2024-04-25T18:49:52.585 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T18:49:52.585 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} --owner scheduled_pdonnell@teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/orig.config.yaml 2024-04-25T18:49:52.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T18:49:52.719 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} email: pdonnell@redhat.com first_in_suite: false job_id: '7673292' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Reduced data availability - Degraded data redundancy sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPt4Wyv06Qc5jTkQ+FBy0+awodsrXtLqkUijs/YZAajDL0HWBWcftf8CvA5gDoQfxUUc2kOdNyPhoD78py8gEc= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWUdEMFgpSjFaq2rLRmc1QqrNOnczHC+/5qujm7oPiKXCCIykkFxtjcKdxGGF83TDxu26e7aeh6KxDNUxhRSPw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T18:49:52.719 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T18:49:52.720 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T18:49:52.720 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T18:49:52.721 INFO:teuthology.task.internal:Checking packages... 2024-04-25T18:49:52.744 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T18:49:52.744 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T18:49:52.744 INFO:teuthology.packaging:ref: None 2024-04-25T18:49:52.744 INFO:teuthology.packaging:tag: None 2024-04-25T18:49:52.744 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T18:49:52.744 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T18:49:52.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T18:49:52.960 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T18:49:52.962 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T18:49:52.968 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T18:49:52.968 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T18:49:52.990 INFO:teuthology.task.internal:Saving configuration 2024-04-25T18:49:53.007 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T18:49:53.013 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T18:49:53.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:44:44.193978', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPt4Wyv06Qc5jTkQ+FBy0+awodsrXtLqkUijs/YZAajDL0HWBWcftf8CvA5gDoQfxUUc2kOdNyPhoD78py8gEc='} 2024-04-25T18:49:53.059 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:44:44.192757', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWUdEMFgpSjFaq2rLRmc1QqrNOnczHC+/5qujm7oPiKXCCIykkFxtjcKdxGGF83TDxu26e7aeh6KxDNUxhRSPw='} 2024-04-25T18:49:53.060 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T18:49:53.065 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T18:49:53.065 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T18:49:53.065 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T18:49:53.140 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f65ffe03f70>, signals=[15]) 2024-04-25T18:49:53.140 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T18:49:53.146 INFO:teuthology.task.internal:Opening connections... 2024-04-25T18:49:53.146 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T18:49:53.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:49:53.224 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T18:49:53.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:49:53.297 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T18:49:53.304 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-25T18:49:53.321 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-25T18:49:53.322 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:49:53.379 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:49:53.380 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:49:53.380 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:49:53.380 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:49:53.380 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-25T18:49:53.614 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-04-25T18:49:53.632 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-04-25T18:49:53.632 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-04-25T18:49:53.689 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:49:53.690 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:49:53.690 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-04-25T18:49:53.705 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T18:49:53.714 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T18:49:53.720 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T18:49:53.720 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:49:53.723 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:49:53.745 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T18:49:53.751 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T18:49:53.751 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:49:53.779 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:49:53.828 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T18:49:53.880 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T18:49:53.897 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T18:49:53.897 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T18:49:53.898 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T18:49:53.898 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:49:53.898 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:49:53.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:49:53.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T18:49:53.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:49:53.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T18:49:54.093 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:49:54.093 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T18:49:54.093 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-04-25T18:49:54.102 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:49:54.102 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T18:49:54.102 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-25T18:49:54.110 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-437.el9.x86_64 2024-04-25T18:49:54.110 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:49:54.111 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T18:49:54.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum upgrade -y linux-firmware 2024-04-25T18:49:54.119 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-04-25T18:49:54.119 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:49:54.119 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T18:49:54.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-04-25T18:49:56.559 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-04-25T18:49:56.696 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-25T18:49:58.518 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T18:49:58.956 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T18:50:02.786 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-25T18:50:03.148 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T18:50:05.234 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-25T18:50:05.804 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T18:50:06.652 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T18:50:07.122 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T18:50:11.677 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-25T18:50:12.138 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T18:50:13.612 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T18:50:13.613 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-25T18:50:13.613 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T18:50:13.720 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:50:13.721 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:50:13.859 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T18:50:13.859 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T18:50:13.862 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-04-25T18:50:14.115 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T18:50:14.117 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-04-25T18:50:14.117 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T18:50:14.212 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T18:50:14.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:50:14.436 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T18:50:14.437 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T18:50:14.439 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi146.front.sepia.ceph.com to complete... 2024-04-25T18:50:39.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25 18:50:39 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T18:50:39.921 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-04-25 18:50:39-- 2024-04-25T18:50:39.921 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 26s 2024-04-25T18:50:39.921 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 341M in 26s (13.2 MB/s) 2024-04-25T18:50:39.924 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:50:39.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T18:50:42.029 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-04-25T18:50:42.232 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-04-25T18:50:42.350 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-04-25T18:50:50.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25 18:50:50 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T18:50:50.250 INFO:teuthology.orchestra.run.smithi146.stderr:FINISHED --2024-04-25 18:50:50-- 2024-04-25T18:50:50.250 INFO:teuthology.orchestra.run.smithi146.stderr:Total wall clock time: 36s 2024-04-25T18:50:50.250 INFO:teuthology.orchestra.run.smithi146.stderr:Downloaded: 1 files, 341M in 35s (9.67 MB/s) 2024-04-25T18:50:50.253 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T18:50:50.253 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T18:50:50.330 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T18:50:52.339 INFO:teuthology.orchestra.run.smithi146.stdout:Verifying... ######################################## 2024-04-25T18:50:52.475 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing... ######################################## 2024-04-25T18:50:52.592 INFO:teuthology.orchestra.run.smithi146.stdout:Updating / installing... 2024-04-25T18:51:00.537 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T18:51:27.095 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T18:51:27.095 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T18:51:27.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T18:51:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T18:51:27.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T18:51:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T18:51:27.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T18:51:27.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T18:51:27.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T18:51:27.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T18:51:27.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T18:51:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T18:51:27.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T18:51:27.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T18:51:27.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T18:51:27.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T18:51:27.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T18:51:27.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T18:51:27.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T18:51:27.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T18:51:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T18:51:27.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T18:51:27.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T18:51:27.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T18:51:27.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T18:51:27.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T18:51:27.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T18:51:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T18:51:27.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T18:51:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T18:51:27.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T18:51:27.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T18:51:27.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T18:51:27.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T18:51:27.204 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T18:51:27.205 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T18:51:27.206 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T18:51:27.207 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T18:51:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T18:51:27.209 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T18:51:27.210 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T18:51:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T18:51:27.212 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T18:51:27.213 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T18:51:27.214 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T18:51:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T18:51:27.216 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T18:51:27.217 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T18:51:27.218 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T18:51:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T18:51:27.220 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T18:51:27.221 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T18:51:27.222 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T18:51:27.223 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T18:51:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T18:51:27.225 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T18:51:27.226 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T18:51:27.227 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T18:51:27.228 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T18:51:27.229 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T18:51:27.230 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T18:51:27.231 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T18:51:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T18:51:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T18:51:27.234 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T18:51:27.235 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T18:51:27.236 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T18:51:27.237 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T18:51:27.238 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T18:51:27.239 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T18:51:27.240 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T18:51:27.241 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T18:51:27.242 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T18:51:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T18:51:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T18:51:27.245 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T18:51:27.246 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T18:51:27.247 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T18:51:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T18:51:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T18:51:27.250 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T18:51:27.251 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T18:51:27.252 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T18:51:27.253 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T18:51:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T18:51:27.255 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T18:51:27.256 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T18:51:27.257 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T18:51:27.258 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T18:51:27.259 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T18:51:27.260 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T18:51:27.261 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T18:51:27.262 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T18:51:27.263 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T18:51:27.264 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T18:51:27.265 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T18:51:27.266 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T18:51:27.267 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T18:51:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T18:51:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T18:51:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T18:51:27.271 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T18:51:27.272 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T18:51:27.273 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T18:51:27.274 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T18:51:27.275 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T18:51:27.276 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T18:51:27.277 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T18:51:27.278 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T18:51:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T18:51:27.280 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T18:51:27.281 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T18:51:27.282 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T18:51:27.283 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T18:51:27.284 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T18:51:27.285 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T18:51:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T18:51:27.287 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T18:51:27.288 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T18:51:27.289 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T18:51:27.290 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T18:51:27.291 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T18:51:27.292 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T18:51:27.293 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T18:51:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T18:51:27.295 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T18:51:27.296 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T18:51:27.297 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T18:51:27.298 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T18:51:27.299 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T18:51:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T18:51:27.301 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T18:51:27.302 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T18:51:27.303 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T18:51:27.304 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T18:51:27.305 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T18:51:27.305 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T18:51:27.305 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T18:51:27.305 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T18:51:27.305 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:51:27.309 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.309 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T18:51:27.331 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T18:51:27.331 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:51:27.331 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:51:27.331 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T18:51:27.331 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T18:51:27.332 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:51:27.333 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.333 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-25T18:51:27.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T18:51:27.405 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T18:51:27.406 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T18:51:27.407 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T18:51:27.408 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:27.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T18:51:27.932 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-04-25T18:51:29.328 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T18:51:29.365 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-04-25T18:51:29.368 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-25T18:51:29.382 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.FEQdDyFAIk 2024-04-25T18:51:29.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FEQdDyFAIk 2024-04-25T18:51:29.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.FEQdDyFAIk 2024-04-25T18:51:29.582 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.FEQdDyFAIk is 6KB 2024-04-25T18:51:29.630 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.FEQdDyFAIk 2024-04-25T18:51:29.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T18:51:29.704 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T18:51:29.704 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T18:51:29.704 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T18:51:29.704 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T18:51:29.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:29.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-04-25T18:51:37.900 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T18:51:37.900 DEBUG:teuthology.orchestra.run.smithi146:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T18:51:37.981 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T18:51:37.982 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T18:51:37.983 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T18:51:37.984 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T18:51:37.985 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T18:51:37.986 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T18:51:37.987 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T18:51:37.988 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T18:51:37.989 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T18:51:37.990 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T18:51:37.991 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T18:51:37.992 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T18:51:37.993 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T18:51:37.994 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T18:51:37.995 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T18:51:37.996 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T18:51:37.997 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T18:51:37.998 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T18:51:37.999 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T18:51:38.000 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T18:51:38.001 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T18:51:38.002 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T18:51:38.003 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T18:51:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T18:51:38.005 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T18:51:38.006 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T18:51:38.007 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T18:51:38.008 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T18:51:38.009 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T18:51:38.010 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T18:51:38.011 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T18:51:38.012 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T18:51:38.013 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T18:51:38.014 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T18:51:38.015 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T18:51:38.016 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T18:51:38.017 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T18:51:38.018 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T18:51:38.019 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T18:51:38.020 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T18:51:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T18:51:38.022 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T18:51:38.023 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T18:51:38.024 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T18:51:38.025 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T18:51:38.026 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T18:51:38.027 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T18:51:38.028 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T18:51:38.029 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T18:51:38.030 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T18:51:38.031 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T18:51:38.032 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T18:51:38.033 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T18:51:38.034 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T18:51:38.035 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T18:51:38.036 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T18:51:38.037 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T18:51:38.038 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T18:51:38.039 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T18:51:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T18:51:38.041 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T18:51:38.042 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T18:51:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T18:51:38.044 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T18:51:38.045 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T18:51:38.046 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T18:51:38.047 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T18:51:38.048 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T18:51:38.049 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T18:51:38.050 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T18:51:38.051 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T18:51:38.052 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T18:51:38.053 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T18:51:38.054 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T18:51:38.055 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T18:51:38.056 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T18:51:38.057 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T18:51:38.058 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T18:51:38.059 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T18:51:38.060 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T18:51:38.061 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T18:51:38.062 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T18:51:38.063 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T18:51:38.064 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T18:51:38.065 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T18:51:38.066 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T18:51:38.067 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T18:51:38.068 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T18:51:38.069 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T18:51:38.070 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T18:51:38.071 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T18:51:38.072 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T18:51:38.073 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T18:51:38.074 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T18:51:38.075 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T18:51:38.076 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T18:51:38.077 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T18:51:38.078 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T18:51:38.079 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T18:51:38.080 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T18:51:38.081 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T18:51:38.082 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T18:51:38.083 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T18:51:38.084 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T18:51:38.085 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T18:51:38.086 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T18:51:38.087 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T18:51:38.088 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T18:51:38.089 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T18:51:38.090 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T18:51:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T18:51:38.092 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T18:51:38.093 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T18:51:38.094 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T18:51:38.095 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T18:51:38.096 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T18:51:38.097 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T18:51:38.098 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T18:51:38.099 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T18:51:38.100 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T18:51:38.101 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T18:51:38.102 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T18:51:38.103 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T18:51:38.104 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T18:51:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T18:51:38.106 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T18:51:38.107 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T18:51:38.108 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T18:51:38.109 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T18:51:38.110 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T18:51:38.111 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T18:51:38.112 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T18:51:38.113 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T18:51:38.114 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T18:51:38.115 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T18:51:38.116 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T18:51:38.117 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T18:51:38.118 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:51:38.122 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:38.122 DEBUG:teuthology.orchestra.run.smithi146:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T18:51:38.144 INFO:teuthology.orchestra.run.smithi146.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T18:51:38.144 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:for file in vmlinuz System.map config; do 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: fi 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:done 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T18:51:38.145 INFO:teuthology.orchestra.run.smithi146.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:38.146 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2024-04-25T18:51:38.146 INFO:teuthology.orchestra.run.smithi146.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T18:51:38.146 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:51:38.146 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:38.146 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2024-04-25T18:51:38.147 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2024-04-25T18:51:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2024-04-25T18:51:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2024-04-25T18:51:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.06 2024-04-25T18:51:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 68.el9 2024-04-25T18:51:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Group : Unspecified 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 8264254 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : builder@centos.org 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2024-04-25T18:51:38.224 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T18:51:38.225 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T18:51:38.226 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T18:51:38.226 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:38.226 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T18:51:38.806 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-04-25T18:51:40.201 INFO:teuthology.orchestra.run.smithi146.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T18:51:40.232 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-04-25T18:51:40.234 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-04-25T18:51:40.249 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.JnepMjVXI2 2024-04-25T18:51:40.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JnepMjVXI2 2024-04-25T18:51:40.313 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.JnepMjVXI2 2024-04-25T18:51:40.450 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.JnepMjVXI2 is 6KB 2024-04-25T18:51:40.498 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.JnepMjVXI2 2024-04-25T18:51:40.513 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T18:51:40.582 INFO:teuthology.orchestra.run.smithi146.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T18:51:40.583 INFO:teuthology.orchestra.run.smithi146.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T18:51:40.583 INFO:teuthology.orchestra.run.smithi146.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T18:51:40.583 INFO:teuthology.orchestra.run.smithi146.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T18:51:40.584 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T18:51:40.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-04-25T18:51:59.892 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T18:51:59.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:51:59.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:52:10.770 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T18:52:10.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:52:10.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:52:18.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T18:52:27.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:52:27.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:52:29.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-04-25T18:52:30.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T18:52:38.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:52:38.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:52:41.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-04-25T18:52:42.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:52:42.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:52:53.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:52:53.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:53:00.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-25T18:53:15.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:53:15.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:53:17.888 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-25T18:53:18.461 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:53:18.462 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T18:53:18.462 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-25T18:53:18.500 INFO:teuthology.orchestra.run.smithi123.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T18:53:18.500 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T18:53:18.501 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T18:53:18.501 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T18:53:19.501 DEBUG:teuthology.parallel:result is None 2024-04-25T18:53:53.161 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T18:54:08.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:54:08.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:54:08.523 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-04-25T18:54:09.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:54:09.156 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T18:54:09.157 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-04-25T18:54:09.209 INFO:teuthology.orchestra.run.smithi146.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T18:54:09.210 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T18:54:09.210 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T18:54:09.210 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T18:54:10.211 DEBUG:teuthology.parallel:result is None 2024-04-25T18:54:10.211 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T18:54:10.223 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T18:54:10.223 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:54:10.233 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:54:10.288 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T18:54:10.295 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T18:54:10.305 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T18:54:10.305 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:54:10.310 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:54:10.386 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T18:54:10.394 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T18:54:10.394 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:54:10.402 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:54:10.540 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:54:10.574 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:54:10.594 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:54:10.635 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:54:10.640 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T18:54:10.647 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T18:54:10.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:54:10.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:54:10.741 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T18:54:10.751 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T18:54:10.751 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:54:10.758 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:54:10.811 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:54:10.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:54:10.999 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:54:11.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:54:11.164 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T18:54:11.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:54:11.245 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:54:11.333 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:54:11.424 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:54:11.506 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:54:11.589 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T18:54:11.590 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:54:11.674 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-25T18:54:11.681 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-04-25T18:54:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:54:11.826 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:54:12.106 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T18:54:12.114 INFO:teuthology.task.internal:Starting timer... 2024-04-25T18:54:12.114 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T18:54:12.139 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T18:54:12.147 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T18:54:12.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-25T18:54:12.246 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-25T18:54:12.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-04-25T18:54:12.370 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-04-25T18:54:12.374 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T18:54:12.374 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-25T18:54:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-25T18:54:12.435 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-04-25T18:54:12.493 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-04-25T18:54:12.494 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2024-04-25T18:54:12.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:54:12.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:54:12.567 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:54:12.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:54:12.637 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T18:54:12.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:54:12.712 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:54:12.807 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T18:54:12.815 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T18:54:12.832 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T18:54:12.833 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T18:59:17.986 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2024-04-25T18:59:17.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:59:17.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:59:18.099 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-25T18:59:18.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-25T18:59:18.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:59:18.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:59:18.352 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-04-25T18:59:18.503 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T18:59:18.503 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T18:59:18.513 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T18:59:18.513 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:59:18.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:59:18.521 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:59:18.521 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:59:18.607 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:59:18.612 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:59:18.646 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:59:18.654 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:59:18.715 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-25T18:59:18.725 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-04-25T18:59:18.748 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-25T18:59:18.756 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-04-25T18:59:18.788 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:59:18.800 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:59:18.828 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:59:18.841 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:59:18.946 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:59:18.954 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:59:18.954 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T18:59:18.954 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.954 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.955 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.955 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.973 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:59:18.981 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:59:18.982 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T18:59:18.982 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.982 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.982 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.982 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:59:18.986 INFO:teuthology.run_tasks:Running task install... 2024-04-25T18:59:18.994 DEBUG:teuthology.task.install:project ceph 2024-04-25T18:59:18.994 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T18:59:18.994 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T18:59:18.994 INFO:teuthology.task.install:Using flavor: default 2024-04-25T18:59:19.000 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T18:59:19.000 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T18:59:19.001 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T18:59:19.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T18:59:19.003 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T18:59:19.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T18:59:19.153 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T18:59:19.153 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T18:59:19.219 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T18:59:19.219 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T18:59:19.352 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:59:19.352 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T18:59:19.352 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:59:19.395 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:59:19.396 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T18:59:19.396 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:59:19.441 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:59:19.441 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:59:19.484 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:59:19.484 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:59:19.574 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:59:19.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:59:19.679 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:59:19.719 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:59:19.754 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-04-25T18:59:19.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-04-25T18:59:19.797 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-25T18:59:19.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-25T18:59:20.077 INFO:teuthology.orchestra.run.smithi146.stdout:48 files removed 2024-04-25T18:59:20.116 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:59:20.124 INFO:teuthology.orchestra.run.smithi123.stdout:48 files removed 2024-04-25T18:59:20.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:59:21.130 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 145 kB/s | 84 kB 00:00 2024-04-25T18:59:21.179 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 144 kB/s | 84 kB 00:00 2024-04-25T18:59:21.897 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-25T18:59:21.938 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-25T18:59:22.547 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-25T18:59:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-25T18:59:23.550 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-25T18:59:23.926 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-04-25T18:59:25.701 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T18:59:26.327 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T18:59:30.494 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-04-25T18:59:31.045 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.2 MB 00:00 2024-04-25T18:59:32.031 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-04-25T18:59:32.765 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-25T18:59:34.196 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T18:59:34.755 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T18:59:40.182 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T18:59:40.724 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-25T18:59:42.576 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:59:42.578 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:59:42.598 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:59:42.599 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:59:42.601 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:59:42.602 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:59:42.604 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:59:42.608 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:59:42.609 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:59:42.679 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T18:59:42.703 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T18:59:42.703 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-25T18:59:42.703 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T18:59:42.703 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-25T18:59:42.703 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T18:59:42.704 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:59:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T18:59:42.706 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T18:59:42.707 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:59:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:59:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:59:42.710 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:59:42.711 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:59:42.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:59:42.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T18:59:42.714 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 4 Packages 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 214 M 2024-04-25T18:59:42.715 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-25T18:59:43.184 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:59:43.186 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:59:43.206 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:59:43.207 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:59:43.209 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:59:43.210 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:59:43.213 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:59:43.216 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:59:43.217 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:59:43.288 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T18:59:43.301 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T18:59:43.301 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T18:59:43.302 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:59:43.303 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-04-25T18:59:43.304 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:59:43.305 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:59:43.306 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:59:43.307 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:59:43.308 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:59:43.309 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:59:43.310 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-04-25T18:59:43.311 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 4 Packages 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 214 M 2024-04-25T18:59:43.312 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-04-25T18:59:43.990 INFO:teuthology.orchestra.run.smithi123.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T18:59:44.143 INFO:teuthology.orchestra.run.smithi146.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-25T18:59:44.184 INFO:teuthology.orchestra.run.smithi123.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 4.3 MB/s | 863 kB 00:00 2024-04-25T18:59:44.243 INFO:teuthology.orchestra.run.smithi123.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.5 MB/s | 147 kB 00:00 2024-04-25T18:59:44.601 INFO:teuthology.orchestra.run.smithi146.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 863 kB 00:00 2024-04-25T18:59:44.668 INFO:teuthology.orchestra.run.smithi123.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 5.3 MB/s | 2.3 MB 00:00 2024-04-25T18:59:44.701 INFO:teuthology.orchestra.run.smithi146.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.4 MB/s | 147 kB 00:00 2024-04-25T18:59:45.003 INFO:teuthology.orchestra.run.smithi123.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-25T18:59:45.177 INFO:teuthology.orchestra.run.smithi123.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-04-25T18:59:45.668 INFO:teuthology.orchestra.run.smithi146.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-04-25T18:59:45.919 INFO:teuthology.orchestra.run.smithi123.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.3 MB/s | 4.9 MB 00:00 2024-04-25T18:59:46.252 INFO:teuthology.orchestra.run.smithi146.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-25T18:59:46.578 INFO:teuthology.orchestra.run.smithi123.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 6.8 MB/s | 20 MB 00:02 2024-04-25T18:59:46.652 INFO:teuthology.orchestra.run.smithi123.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 341 kB/s | 25 kB 00:00 2024-04-25T18:59:46.777 INFO:teuthology.orchestra.run.smithi146.stdout:(6/152): ceph-common-19.0.0-3291.g994e2f20.el9. 6.7 MB/s | 20 MB 00:02 2024-04-25T18:59:47.302 INFO:teuthology.orchestra.run.smithi146.stdout:(7/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T18:59:47.536 INFO:teuthology.orchestra.run.smithi123.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 6.3 MB/s | 10 MB 00:01 2024-04-25T18:59:47.636 INFO:teuthology.orchestra.run.smithi123.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.2 MB/s | 224 kB 00:00 2024-04-25T18:59:47.729 INFO:teuthology.orchestra.run.smithi123.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 297 kB/s | 27 kB 00:00 2024-04-25T18:59:47.812 INFO:teuthology.orchestra.run.smithi123.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 385 kB/s | 32 kB 00:00 2024-04-25T18:59:47.910 INFO:teuthology.orchestra.run.smithi146.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 4.9 MB 00:01 2024-04-25T18:59:47.979 INFO:teuthology.orchestra.run.smithi123.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.5 MB/s | 756 kB 00:00 2024-04-25T18:59:48.019 INFO:teuthology.orchestra.run.smithi146.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 234 kB/s | 25 kB 00:00 2024-04-25T18:59:48.103 INFO:teuthology.orchestra.run.smithi123.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.3 MB/s | 158 kB 00:00 2024-04-25T18:59:48.196 INFO:teuthology.orchestra.run.smithi123.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.4 MB/s | 127 kB 00:00 2024-04-25T18:59:48.371 INFO:teuthology.orchestra.run.smithi123.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.8 MB/s | 497 kB 00:00 2024-04-25T18:59:49.178 INFO:teuthology.orchestra.run.smithi146.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 7.4 MB/s | 18 MB 00:02 2024-04-25T18:59:49.230 INFO:teuthology.orchestra.run.smithi123.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.0 MB/s | 5.2 MB 00:00 2024-04-25T18:59:49.267 INFO:teuthology.orchestra.run.smithi146.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.5 MB/s | 224 kB 00:00 2024-04-25T18:59:49.289 INFO:teuthology.orchestra.run.smithi123.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 790 kB/s | 45 kB 00:00 2024-04-25T18:59:49.345 INFO:teuthology.orchestra.run.smithi146.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 353 kB/s | 27 kB 00:00 2024-04-25T18:59:49.380 INFO:teuthology.orchestra.run.smithi123.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.5 MB/s | 135 kB 00:00 2024-04-25T18:59:49.437 INFO:teuthology.orchestra.run.smithi146.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 350 kB/s | 32 kB 00:00 2024-04-25T18:59:49.456 INFO:teuthology.orchestra.run.smithi123.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.1 MB/s | 161 kB 00:00 2024-04-25T18:59:49.596 INFO:teuthology.orchestra.run.smithi146.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.7 MB/s | 756 kB 00:00 2024-04-25T18:59:49.671 INFO:teuthology.orchestra.run.smithi146.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T18:59:49.763 INFO:teuthology.orchestra.run.smithi146.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.4 MB/s | 127 kB 00:00 2024-04-25T18:59:49.764 INFO:teuthology.orchestra.run.smithi123.stdout:(22/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.9 MB/s | 18 MB 00:04 2024-04-25T18:59:49.806 INFO:teuthology.orchestra.run.smithi123.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 918 kB/s | 321 kB 00:00 2024-04-25T18:59:49.863 INFO:teuthology.orchestra.run.smithi146.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 4.9 MB/s | 497 kB 00:00 2024-04-25T18:59:49.873 INFO:teuthology.orchestra.run.smithi123.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T18:59:49.907 INFO:teuthology.orchestra.run.smithi123.stdout:(25/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.1 MB/s | 298 kB 00:00 2024-04-25T18:59:49.949 INFO:teuthology.orchestra.run.smithi123.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T18:59:50.040 INFO:teuthology.orchestra.run.smithi123.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 169 kB 00:00 2024-04-25T18:59:50.124 INFO:teuthology.orchestra.run.smithi123.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 414 kB/s | 34 kB 00:00 2024-04-25T18:59:50.216 INFO:teuthology.orchestra.run.smithi123.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.5 MB/s | 142 kB 00:00 2024-04-25T18:59:50.567 INFO:teuthology.orchestra.run.smithi123.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.6 MB/s | 3.0 MB 00:00 2024-04-25T18:59:50.691 INFO:teuthology.orchestra.run.smithi146.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.2 MB/s | 5.2 MB 00:00 2024-04-25T18:59:50.788 INFO:teuthology.orchestra.run.smithi146.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 472 kB/s | 45 kB 00:00 2024-04-25T18:59:50.847 INFO:teuthology.orchestra.run.smithi146.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T18:59:50.850 INFO:teuthology.orchestra.run.smithi123.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 5.6 MB/s | 3.5 MB 00:00 2024-04-25T18:59:50.914 INFO:teuthology.orchestra.run.smithi146.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.4 MB/s | 161 kB 00:00 2024-04-25T18:59:50.942 INFO:teuthology.orchestra.run.smithi123.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.7 MB/s | 249 kB 00:00 2024-04-25T18:59:51.023 INFO:teuthology.orchestra.run.smithi146.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.9 MB/s | 321 kB 00:00 2024-04-25T18:59:51.042 INFO:teuthology.orchestra.run.smithi123.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 498 kB/s | 49 kB 00:00 2024-04-25T18:59:51.082 INFO:teuthology.orchestra.run.smithi146.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 5.1 MB/s | 298 kB 00:00 2024-04-25T18:59:51.117 INFO:teuthology.orchestra.run.smithi123.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 222 kB/s | 16 kB 00:00 2024-04-25T18:59:51.182 INFO:teuthology.orchestra.run.smithi146.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.0 MB/s | 100 kB 00:00 2024-04-25T18:59:51.218 INFO:teuthology.orchestra.run.smithi123.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 273 kB 00:00 2024-04-25T18:59:51.257 INFO:teuthology.orchestra.run.smithi146.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T18:59:51.393 INFO:teuthology.orchestra.run.smithi123.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.3 MB/s | 762 kB 00:00 2024-04-25T18:59:51.477 INFO:teuthology.orchestra.run.smithi123.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 220 kB/s | 18 kB 00:00 2024-04-25T18:59:51.716 INFO:teuthology.orchestra.run.smithi146.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 6.7 MB/s | 3.0 MB 00:00 2024-04-25T18:59:51.874 INFO:teuthology.orchestra.run.smithi146.stdout:(27/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.2 MB/s | 10 MB 00:04 2024-04-25T18:59:51.908 INFO:teuthology.orchestra.run.smithi146.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 888 kB/s | 169 kB 00:00 2024-04-25T18:59:51.950 INFO:teuthology.orchestra.run.smithi146.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 462 kB/s | 34 kB 00:00 2024-04-25T18:59:51.983 INFO:teuthology.orchestra.run.smithi146.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.9 MB/s | 142 kB 00:00 2024-04-25T18:59:52.160 INFO:teuthology.orchestra.run.smithi123.stdout:(38/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 7.8 MB/s | 43 MB 00:05 2024-04-25T18:59:52.311 INFO:teuthology.orchestra.run.smithi123.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-04-25T18:59:52.336 INFO:teuthology.orchestra.run.smithi123.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 94 kB/s | 80 kB 00:00 2024-04-25T18:59:52.542 INFO:teuthology.orchestra.run.smithi146.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-25T18:59:52.601 INFO:teuthology.orchestra.run.smithi146.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 4.3 MB/s | 249 kB 00:00 2024-04-25T18:59:52.651 INFO:teuthology.orchestra.run.smithi146.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 1.0 MB/s | 49 kB 00:00 2024-04-25T18:59:52.693 INFO:teuthology.orchestra.run.smithi146.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 402 kB/s | 16 kB 00:00 2024-04-25T18:59:52.752 INFO:teuthology.orchestra.run.smithi146.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.7 MB/s | 273 kB 00:00 2024-04-25T18:59:52.852 INFO:teuthology.orchestra.run.smithi146.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 7.5 MB/s | 762 kB 00:00 2024-04-25T18:59:52.870 INFO:teuthology.orchestra.run.smithi123.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T18:59:52.894 INFO:teuthology.orchestra.run.smithi146.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 448 kB/s | 18 kB 00:00 2024-04-25T18:59:52.920 INFO:teuthology.orchestra.run.smithi123.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 321 kB/s | 195 kB 00:00 2024-04-25T18:59:53.062 INFO:teuthology.orchestra.run.smithi123.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 349 kB/s | 253 kB 00:00 2024-04-25T18:59:53.202 INFO:teuthology.orchestra.run.smithi146.stdout:(38/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 2.8 MB/s | 3.5 MB 00:01 2024-04-25T18:59:53.228 INFO:teuthology.orchestra.run.smithi146.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 241 kB/s | 80 kB 00:00 2024-04-25T18:59:53.419 INFO:teuthology.orchestra.run.smithi146.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-25T18:59:53.453 INFO:teuthology.orchestra.run.smithi146.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 870 kB/s | 195 kB 00:00 2024-04-25T18:59:53.487 INFO:teuthology.orchestra.run.smithi146.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-04-25T18:59:53.528 INFO:teuthology.orchestra.run.smithi123.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 175 kB/s | 106 kB 00:00 2024-04-25T18:59:53.595 INFO:teuthology.orchestra.run.smithi146.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 8.8 MB/s | 1.2 MB 00:00 2024-04-25T18:59:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 250 kB/s | 135 kB 00:00 2024-04-25T18:59:53.620 INFO:teuthology.orchestra.run.smithi146.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 834 kB/s | 106 kB 00:00 2024-04-25T18:59:53.654 INFO:teuthology.orchestra.run.smithi146.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T18:59:53.679 INFO:teuthology.orchestra.run.smithi146.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T18:59:53.687 INFO:teuthology.orchestra.run.smithi123.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-04-25T18:59:53.721 INFO:teuthology.orchestra.run.smithi146.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-25T18:59:53.754 INFO:teuthology.orchestra.run.smithi123.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 556 kB/s | 125 kB 00:00 2024-04-25T18:59:53.780 INFO:teuthology.orchestra.run.smithi146.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.8 MB/s | 689 kB 00:00 2024-04-25T18:59:53.821 INFO:teuthology.orchestra.run.smithi123.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 997 kB/s | 215 kB 00:00 2024-04-25T18:59:53.839 INFO:teuthology.orchestra.run.smithi146.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-25T18:59:53.864 INFO:teuthology.orchestra.run.smithi146.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-25T18:59:53.940 INFO:teuthology.orchestra.run.smithi146.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 218 kB/s | 21 kB 00:00 2024-04-25T18:59:54.002 INFO:teuthology.orchestra.run.smithi146.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-25T18:59:54.048 INFO:teuthology.orchestra.run.smithi146.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 311 kB/s | 33 kB 00:00 2024-04-25T18:59:54.072 INFO:teuthology.orchestra.run.smithi123.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-25T18:59:54.274 INFO:teuthology.orchestra.run.smithi146.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T18:59:54.316 INFO:teuthology.orchestra.run.smithi146.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 69 kB/s | 18 kB 00:00 2024-04-25T18:59:54.397 INFO:teuthology.orchestra.run.smithi123.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 186 kB/s | 107 kB 00:00 2024-04-25T18:59:54.407 INFO:teuthology.orchestra.run.smithi146.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 11 kB 00:00 2024-04-25T18:59:54.441 INFO:teuthology.orchestra.run.smithi146.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-25T18:59:54.447 INFO:teuthology.orchestra.run.smithi123.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-04-25T18:59:54.472 INFO:teuthology.orchestra.run.smithi123.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 54 kB/s | 21 kB 00:00 2024-04-25T18:59:54.508 INFO:teuthology.orchestra.run.smithi146.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 462 kB/s | 45 kB 00:00 2024-04-25T18:59:54.539 INFO:teuthology.orchestra.run.smithi123.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-25T18:59:54.550 INFO:teuthology.orchestra.run.smithi146.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-25T18:59:54.564 INFO:teuthology.orchestra.run.smithi123.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 282 kB/s | 33 kB 00:00 2024-04-25T18:59:54.592 INFO:teuthology.orchestra.run.smithi146.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 582 kB 00:00 2024-04-25T18:59:54.634 INFO:teuthology.orchestra.run.smithi146.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 326 kB/s | 25 kB 00:00 2024-04-25T18:59:54.676 INFO:teuthology.orchestra.run.smithi146.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 575 kB/s | 47 kB 00:00 2024-04-25T18:59:54.710 INFO:teuthology.orchestra.run.smithi146.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.9 MB/s | 292 kB 00:00 2024-04-25T18:59:54.743 INFO:teuthology.orchestra.run.smithi146.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2024-04-25T18:59:54.785 INFO:teuthology.orchestra.run.smithi146.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 485 kB/s | 35 kB 00:00 2024-04-25T18:59:54.865 INFO:teuthology.orchestra.run.smithi123.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 56 kB/s | 18 kB 00:00 2024-04-25T18:59:54.890 INFO:teuthology.orchestra.run.smithi123.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 34 kB/s | 11 kB 00:00 2024-04-25T18:59:55.077 INFO:teuthology.orchestra.run.smithi146.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-25T18:59:55.140 INFO:teuthology.orchestra.run.smithi123.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-04-25T18:59:55.166 INFO:teuthology.orchestra.run.smithi123.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 167 kB/s | 45 kB 00:00 2024-04-25T18:59:55.191 INFO:teuthology.orchestra.run.smithi123.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 539 kB/s | 160 kB 00:00 2024-04-25T18:59:55.253 INFO:teuthology.orchestra.run.smithi146.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-25T18:59:55.311 INFO:teuthology.orchestra.run.smithi146.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.2 MB/s | 1.6 MB 00:00 2024-04-25T18:59:55.313 INFO:teuthology.orchestra.run.smithi123.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.8 MB/s | 662 kB 00:00 2024-04-25T18:59:55.353 INFO:teuthology.orchestra.run.smithi146.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 9.9 MB/s | 911 kB 00:00 2024-04-25T18:59:55.358 INFO:teuthology.orchestra.run.smithi123.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-25T18:59:55.378 INFO:teuthology.orchestra.run.smithi146.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T18:59:55.383 INFO:teuthology.orchestra.run.smithi123.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 133 kB/s | 25 kB 00:00 2024-04-25T18:59:55.484 INFO:teuthology.orchestra.run.smithi123.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 288 kB/s | 47 kB 00:00 2024-04-25T18:59:55.517 INFO:teuthology.orchestra.run.smithi123.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 292 kB 00:00 2024-04-25T18:59:55.542 INFO:teuthology.orchestra.run.smithi123.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-25T18:59:55.567 INFO:teuthology.orchestra.run.smithi123.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 448 kB/s | 35 kB 00:00 2024-04-25T18:59:55.720 INFO:teuthology.orchestra.run.smithi146.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T18:59:55.779 INFO:teuthology.orchestra.run.smithi146.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-04-25T18:59:55.804 INFO:teuthology.orchestra.run.smithi146.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-25T18:59:56.251 INFO:teuthology.orchestra.run.smithi123.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-25T18:59:56.288 INFO:teuthology.orchestra.run.smithi146.stdout:(74/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 5.2 MB/s | 43 MB 00:08 2024-04-25T18:59:56.385 INFO:teuthology.orchestra.run.smithi123.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-04-25T18:59:56.488 INFO:teuthology.orchestra.run.smithi146.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.4 MB/s | 6.0 MB 00:00 2024-04-25T18:59:56.522 INFO:teuthology.orchestra.run.smithi146.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 340 kB/s | 244 kB 00:00 2024-04-25T18:59:56.547 INFO:teuthology.orchestra.run.smithi146.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-04-25T18:59:56.560 INFO:teuthology.orchestra.run.smithi123.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 4.7 MB/s | 4.7 MB 00:01 2024-04-25T18:59:56.572 INFO:teuthology.orchestra.run.smithi146.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 326 kB/s | 26 kB 00:00 2024-04-25T18:59:56.597 INFO:teuthology.orchestra.run.smithi146.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 393 kB/s | 19 kB 00:00 2024-04-25T18:59:56.602 INFO:teuthology.orchestra.run.smithi123.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 2.6 MB/s | 911 kB 00:00 2024-04-25T18:59:56.623 INFO:teuthology.orchestra.run.smithi146.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-25T18:59:56.627 INFO:teuthology.orchestra.run.smithi123.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 584 kB/s | 133 kB 00:00 2024-04-25T18:59:56.648 INFO:teuthology.orchestra.run.smithi146.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 477 kB/s | 35 kB 00:00 2024-04-25T18:59:56.761 INFO:teuthology.orchestra.run.smithi123.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 705 kB/s | 87 kB 00:00 2024-04-25T18:59:56.807 INFO:teuthology.orchestra.run.smithi146.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-25T18:59:56.878 INFO:teuthology.orchestra.run.smithi123.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-04-25T18:59:56.957 INFO:teuthology.orchestra.run.smithi146.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T18:59:56.990 INFO:teuthology.orchestra.run.smithi146.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 653 kB/s | 222 kB 00:00 2024-04-25T18:59:57.016 INFO:teuthology.orchestra.run.smithi146.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 393 kB/s | 77 kB 00:00 2024-04-25T18:59:57.036 INFO:teuthology.orchestra.run.smithi123.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-25T18:59:57.041 INFO:teuthology.orchestra.run.smithi146.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 526 kB/s | 42 kB 00:00 2024-04-25T18:59:57.091 INFO:teuthology.orchestra.run.smithi146.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.6 MB/s | 269 kB 00:00 2024-04-25T18:59:57.124 INFO:teuthology.orchestra.run.smithi146.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-25T18:59:57.128 INFO:teuthology.orchestra.run.smithi123.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-25T18:59:57.149 INFO:teuthology.orchestra.run.smithi146.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T18:59:57.175 INFO:teuthology.orchestra.run.smithi146.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 644 kB/s | 51 kB 00:00 2024-04-25T18:59:57.209 INFO:teuthology.orchestra.run.smithi146.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 938 kB/s | 54 kB 00:00 2024-04-25T18:59:57.366 INFO:teuthology.orchestra.run.smithi123.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-25T18:59:57.395 INFO:teuthology.orchestra.run.smithi123.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 720 kB/s | 248 kB 00:00 2024-04-25T18:59:57.421 INFO:teuthology.orchestra.run.smithi123.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 90 kB/s | 26 kB 00:00 2024-04-25T18:59:57.446 INFO:teuthology.orchestra.run.smithi123.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 250 kB/s | 19 kB 00:00 2024-04-25T18:59:57.471 INFO:teuthology.orchestra.run.smithi123.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 715 kB/s | 35 kB 00:00 2024-04-25T18:59:57.501 INFO:teuthology.orchestra.run.smithi146.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 128 kB/s | 42 kB 00:00 2024-04-25T18:59:57.504 INFO:teuthology.orchestra.run.smithi123.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T18:59:57.526 INFO:teuthology.orchestra.run.smithi146.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 142 kB/s | 44 kB 00:00 2024-04-25T18:59:57.605 INFO:teuthology.orchestra.run.smithi123.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 250 kB 00:00 2024-04-25T18:59:57.705 INFO:teuthology.orchestra.run.smithi123.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-25T18:59:57.805 INFO:teuthology.orchestra.run.smithi123.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-25T18:59:57.830 INFO:teuthology.orchestra.run.smithi123.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 360 kB/s | 77 kB 00:00 2024-04-25T18:59:57.847 INFO:teuthology.orchestra.run.smithi123.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 322 kB/s | 42 kB 00:00 2024-04-25T18:59:57.868 INFO:teuthology.orchestra.run.smithi146.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T18:59:57.898 INFO:teuthology.orchestra.run.smithi123.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-25T18:59:57.902 INFO:teuthology.orchestra.run.smithi146.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 552 kB/s | 221 kB 00:00 2024-04-25T18:59:57.918 INFO:teuthology.orchestra.run.smithi146.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 96 kB/s | 36 kB 00:00 2024-04-25T18:59:57.931 INFO:teuthology.orchestra.run.smithi123.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 2.7 MB/s | 269 kB 00:00 2024-04-25T18:59:57.935 INFO:teuthology.orchestra.run.smithi146.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 739 kB/s | 47 kB 00:00 2024-04-25T18:59:57.956 INFO:teuthology.orchestra.run.smithi123.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T18:59:57.969 INFO:teuthology.orchestra.run.smithi146.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.1 MB/s | 305 kB 00:00 2024-04-25T18:59:57.981 INFO:teuthology.orchestra.run.smithi123.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 649 kB/s | 51 kB 00:00 2024-04-25T18:59:57.986 INFO:teuthology.orchestra.run.smithi146.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 884 kB/s | 58 kB 00:00 2024-04-25T18:59:58.003 INFO:teuthology.orchestra.run.smithi146.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-25T18:59:58.015 INFO:teuthology.orchestra.run.smithi123.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 945 kB/s | 54 kB 00:00 2024-04-25T18:59:58.028 INFO:teuthology.orchestra.run.smithi146.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T18:59:58.045 INFO:teuthology.orchestra.run.smithi146.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:59:58.057 INFO:teuthology.orchestra.run.smithi123.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 558 kB/s | 42 kB 00:00 2024-04-25T18:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 392 kB/s | 22 kB 00:00 2024-04-25T18:59:58.121 INFO:teuthology.orchestra.run.smithi146.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.3 MB/s | 863 kB 00:00 2024-04-25T18:59:58.149 INFO:teuthology.orchestra.run.smithi123.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 339 kB/s | 44 kB 00:00 2024-04-25T18:59:58.271 INFO:teuthology.orchestra.run.smithi146.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-25T18:59:58.274 INFO:teuthology.orchestra.run.smithi123.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-04-25T18:59:58.371 INFO:teuthology.orchestra.run.smithi146.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-25T18:59:58.405 INFO:teuthology.orchestra.run.smithi146.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 923 kB/s | 309 kB 00:00 2024-04-25T18:59:58.408 INFO:teuthology.orchestra.run.smithi123.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 144 kB/s | 36 kB 00:00 2024-04-25T18:59:58.430 INFO:teuthology.orchestra.run.smithi146.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 449 kB/s | 26 kB 00:00 2024-04-25T18:59:58.464 INFO:teuthology.orchestra.run.smithi146.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 837 kB/s | 49 kB 00:00 2024-04-25T18:59:58.489 INFO:teuthology.orchestra.run.smithi146.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T18:59:58.531 INFO:teuthology.orchestra.run.smithi146.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T18:59:58.598 INFO:teuthology.orchestra.run.smithi146.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-25T18:59:58.656 INFO:teuthology.orchestra.run.smithi146.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 758 kB/s | 43 kB 00:00 2024-04-25T18:59:58.690 INFO:teuthology.orchestra.run.smithi146.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-25T18:59:58.691 INFO:teuthology.orchestra.run.smithi123.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-25T18:59:58.715 INFO:teuthology.orchestra.run.smithi146.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 559 kB/s | 32 kB 00:00 2024-04-25T18:59:58.725 INFO:teuthology.orchestra.run.smithi123.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 109 kB/s | 47 kB 00:00 2024-04-25T18:59:58.749 INFO:teuthology.orchestra.run.smithi146.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 258 kB/s | 15 kB 00:00 2024-04-25T18:59:58.750 INFO:teuthology.orchestra.run.smithi123.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 925 kB/s | 305 kB 00:00 2024-04-25T18:59:58.775 INFO:teuthology.orchestra.run.smithi123.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 724 kB/s | 58 kB 00:00 2024-04-25T18:59:58.811 INFO:teuthology.orchestra.run.smithi146.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T18:59:58.817 INFO:teuthology.orchestra.run.smithi123.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-25T18:59:58.843 INFO:teuthology.orchestra.run.smithi123.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 705 kB/s | 64 kB 00:00 2024-04-25T18:59:58.849 INFO:teuthology.orchestra.run.smithi146.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-25T18:59:58.868 INFO:teuthology.orchestra.run.smithi123.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 489 kB/s | 45 kB 00:00 2024-04-25T18:59:58.883 INFO:teuthology.orchestra.run.smithi146.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.2 MB/s | 303 kB 00:00 2024-04-25T18:59:58.925 INFO:teuthology.orchestra.run.smithi146.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-25T18:59:58.926 INFO:teuthology.orchestra.run.smithi123.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 272 kB/s | 22 kB 00:00 2024-04-25T18:59:58.958 INFO:teuthology.orchestra.run.smithi146.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T18:59:58.977 INFO:teuthology.orchestra.run.smithi123.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.4 MB/s | 863 kB 00:00 2024-04-25T18:59:59.017 INFO:teuthology.orchestra.run.smithi146.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T18:59:59.092 INFO:teuthology.orchestra.run.smithi146.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T18:59:59.134 INFO:teuthology.orchestra.run.smithi146.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T18:59:59.176 INFO:teuthology.orchestra.run.smithi146.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 213 kB/s | 18 kB 00:00 2024-04-25T18:59:59.218 INFO:teuthology.orchestra.run.smithi146.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-25T18:59:59.260 INFO:teuthology.orchestra.run.smithi146.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-25T18:59:59.327 INFO:teuthology.orchestra.run.smithi146.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-25T18:59:59.453 INFO:teuthology.orchestra.run.smithi146.stdout:(129/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-04-25T18:59:59.494 INFO:teuthology.orchestra.run.smithi146.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-25T18:59:59.545 INFO:teuthology.orchestra.run.smithi146.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 214 kB/s | 46 kB 00:00 2024-04-25T18:59:59.570 INFO:teuthology.orchestra.run.smithi146.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 678 kB/s | 79 kB 00:00 2024-04-25T18:59:59.620 INFO:teuthology.orchestra.run.smithi146.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 463 kB/s | 58 kB 00:00 2024-04-25T18:59:59.671 INFO:teuthology.orchestra.run.smithi146.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-25T18:59:59.696 INFO:teuthology.orchestra.run.smithi146.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T18:59:59.727 INFO:teuthology.orchestra.run.smithi123.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 31 kB/s | 19 kB 00:00 2024-04-25T18:59:59.729 INFO:teuthology.orchestra.run.smithi146.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-25T18:59:59.805 INFO:teuthology.orchestra.run.smithi146.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 212 kB/s | 28 kB 00:00 2024-04-25T18:59:59.855 INFO:teuthology.orchestra.run.smithi146.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2024-04-25T18:59:59.906 INFO:teuthology.orchestra.run.smithi146.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-04-25T18:59:59.931 INFO:teuthology.orchestra.run.smithi146.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 481 kB/s | 36 kB 00:00 2024-04-25T18:59:59.956 INFO:teuthology.orchestra.run.smithi146.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 836 kB/s | 188 kB 00:00 2024-04-25T18:59:59.981 INFO:teuthology.orchestra.run.smithi146.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 686 kB/s | 51 kB 00:00 2024-04-25T19:00:00.015 INFO:teuthology.orchestra.run.smithi146.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T19:00:00.049 INFO:teuthology.orchestra.run.smithi146.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-25T19:00:00.052 INFO:teuthology.orchestra.run.smithi123.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 276 kB/s | 309 kB 00:01 2024-04-25T19:00:00.082 INFO:teuthology.orchestra.run.smithi146.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 303 kB/s | 20 kB 00:00 2024-04-25T19:00:00.124 INFO:teuthology.orchestra.run.smithi146.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-25T19:00:00.158 INFO:teuthology.orchestra.run.smithi146.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T19:00:00.186 INFO:teuthology.orchestra.run.smithi123.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-04-25T19:00:00.244 INFO:teuthology.orchestra.run.smithi146.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.9 MB/s | 1.6 MB 00:00 2024-04-25T19:00:00.311 INFO:teuthology.orchestra.run.smithi123.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 391 kB/s | 49 kB 00:00 2024-04-25T19:00:00.345 INFO:teuthology.orchestra.run.smithi123.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 371 kB/s | 548 kB 00:01 2024-04-25T19:00:00.358 INFO:teuthology.orchestra.run.smithi146.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T19:00:00.508 INFO:teuthology.orchestra.run.smithi146.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T19:00:00.585 INFO:teuthology.orchestra.run.smithi123.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 247 kB/s | 67 kB 00:00 2024-04-25T19:00:00.738 INFO:teuthology.orchestra.run.smithi123.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 386 kB/s | 151 kB 00:00 2024-04-25T19:00:00.934 INFO:teuthology.orchestra.run.smithi146.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 4.1 MB/s | 3.3 MB 00:00 2024-04-25T19:00:01.342 INFO:teuthology.orchestra.run.smithi146.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-25T19:00:01.351 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:00:01.351 INFO:teuthology.orchestra.run.smithi146.stdout:Total 12 MB/s | 214 MB 00:18 2024-04-25T19:00:01.579 INFO:teuthology.orchestra.run.smithi123.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 652 kB/s | 548 kB 00:00 2024-04-25T19:00:01.696 INFO:teuthology.orchestra.run.smithi123.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 374 kB/s | 43 kB 00:00 2024-04-25T19:00:01.821 INFO:teuthology.orchestra.run.smithi123.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 678 kB/s | 838 kB 00:01 2024-04-25T19:00:01.847 INFO:teuthology.orchestra.run.smithi123.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-04-25T19:00:01.939 INFO:teuthology.orchestra.run.smithi123.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-25T19:00:02.147 INFO:teuthology.orchestra.run.smithi123.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 573 kB/s | 172 kB 00:00 2024-04-25T19:00:02.564 INFO:teuthology.orchestra.run.smithi123.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 574 kB/s | 358 kB 00:00 2024-04-25T19:00:02.656 INFO:teuthology.orchestra.run.smithi123.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 597 kB/s | 303 kB 00:00 2024-04-25T19:00:02.698 INFO:teuthology.orchestra.run.smithi123.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-04-25T19:00:03.073 INFO:teuthology.orchestra.run.smithi123.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 494 kB/s | 206 kB 00:00 2024-04-25T19:00:03.357 INFO:teuthology.orchestra.run.smithi123.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 511 kB/s | 144 kB 00:00 2024-04-25T19:00:03.474 INFO:teuthology.orchestra.run.smithi123.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T19:00:03.591 INFO:teuthology.orchestra.run.smithi123.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-25T19:00:03.699 INFO:teuthology.orchestra.run.smithi123.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-04-25T19:00:03.800 INFO:teuthology.orchestra.run.smithi123.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-04-25T19:00:03.900 INFO:teuthology.orchestra.run.smithi123.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-04-25T19:00:04.278 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-04-25T19:00:04.533 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-04-25T19:00:04.533 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi123.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 780 kB/s | 1.0 MB 00:01 2024-04-25T19:00:05.342 INFO:teuthology.orchestra.run.smithi123.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 465 kB/s | 46 kB 00:00 2024-04-25T19:00:05.400 INFO:teuthology.orchestra.run.smithi123.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 772 kB/s | 2.0 MB 00:02 2024-04-25T19:00:05.534 INFO:teuthology.orchestra.run.smithi123.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 412 kB/s | 79 kB 00:00 2024-04-25T19:00:05.762 INFO:teuthology.orchestra.run.smithi123.stdout:(132/152): libarrow-9.0.0-10.el9.x86_64.rpm 750 kB/s | 4.4 MB 00:06 2024-04-25T19:00:05.787 INFO:teuthology.orchestra.run.smithi123.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 150 kB/s | 58 kB 00:00 2024-04-25T19:00:05.870 INFO:teuthology.orchestra.run.smithi123.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-04-25T19:00:05.979 INFO:teuthology.orchestra.run.smithi123.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 470 kB/s | 90 kB 00:00 2024-04-25T19:00:06.004 INFO:teuthology.orchestra.run.smithi123.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 212 kB/s | 28 kB 00:00 2024-04-25T19:00:06.030 INFO:teuthology.orchestra.run.smithi123.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 550 kB/s | 272 kB 00:00 2024-04-25T19:00:06.113 INFO:teuthology.orchestra.run.smithi123.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 231 kB/s | 31 kB 00:00 2024-04-25T19:00:06.222 INFO:teuthology.orchestra.run.smithi123.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2024-04-25T19:00:06.247 INFO:teuthology.orchestra.run.smithi123.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 269 kB/s | 36 kB 00:00 2024-04-25T19:00:06.331 INFO:teuthology.orchestra.run.smithi123.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 473 kB/s | 51 kB 00:00 2024-04-25T19:00:06.356 INFO:teuthology.orchestra.run.smithi123.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 538 kB/s | 188 kB 00:00 2024-04-25T19:00:06.514 INFO:teuthology.orchestra.run.smithi123.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 491 kB/s | 90 kB 00:00 2024-04-25T19:00:06.623 INFO:teuthology.orchestra.run.smithi123.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 612 kB/s | 230 kB 00:00 2024-04-25T19:00:06.648 INFO:teuthology.orchestra.run.smithi123.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-04-25T19:00:06.973 INFO:teuthology.orchestra.run.smithi123.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 548 kB/s | 192 kB 00:00 2024-04-25T19:00:07.007 INFO:teuthology.orchestra.run.smithi123.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 657 kB/s | 427 kB 00:00 2024-04-25T19:00:07.262 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-04-25T19:00:07.262 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-04-25T19:00:07.324 INFO:teuthology.orchestra.run.smithi123.stdout:(148/152): librados2-19.0.0-3291.g994e2f20.el9. 9.5 MB/s | 3.3 MB 00:00 2024-04-25T19:00:07.433 INFO:teuthology.orchestra.run.smithi123.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-25T19:00:07.582 INFO:teuthology.orchestra.run.smithi123.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T19:00:07.753 INFO:teuthology.orchestra.run.smithi123.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-25T19:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 750 kB/s | 1.6 MB 00:02 2024-04-25T19:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout:Total 8.2 MB/s | 214 MB 00:26 2024-04-25T19:00:11.646 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-25T19:00:11.883 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-25T19:00:11.884 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-25T19:00:13.702 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-04-25T19:00:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T19:00:13.995 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T19:00:14.167 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T19:00:14.239 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T19:00:14.276 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T19:00:14.414 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T19:00:14.601 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-25T19:00:14.602 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-25T19:00:14.853 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T19:00:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:00:15.168 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:00:15.199 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:00:15.339 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:00:15.373 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:00:15.703 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T19:00:15.772 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T19:00:15.795 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:00:15.946 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:00:15.971 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:00:16.074 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:00:16.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:00:16.180 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:00:16.492 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:00:16.573 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:00:16.763 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T19:00:16.895 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T19:00:16.939 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T19:00:16.987 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T19:00:17.101 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T19:00:17.151 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T19:00:17.217 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T19:00:17.290 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T19:00:17.332 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T19:00:17.455 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T19:00:17.538 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T19:00:17.575 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T19:00:17.636 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T19:00:17.749 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T19:00:17.786 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T19:00:17.831 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T19:00:17.867 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T19:00:17.981 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T19:00:18.045 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T19:00:18.153 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T19:00:18.344 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T19:00:18.416 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T19:00:19.076 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T19:00:19.180 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T19:00:19.223 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T19:00:19.268 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T19:00:19.308 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T19:00:19.367 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T19:00:19.622 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T19:00:19.676 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T19:00:19.752 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T19:00:19.846 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T19:00:19.893 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T19:00:19.932 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T19:00:20.000 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T19:00:20.039 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T19:00:20.101 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T19:00:20.159 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T19:00:20.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T19:00:20.421 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T19:00:20.459 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T19:00:20.516 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T19:00:20.654 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T19:00:20.700 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T19:00:20.748 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T19:00:20.806 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T19:00:20.840 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T19:00:20.956 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-25T19:00:21.024 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T19:00:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T19:00:21.202 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T19:00:21.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T19:00:21.274 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T19:00:21.312 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T19:00:21.319 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T19:00:21.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T19:00:21.373 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T19:00:21.408 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T19:00:21.444 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T19:00:21.500 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T19:00:21.550 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T19:00:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T19:00:21.596 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T19:00:21.719 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T19:00:21.775 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T19:00:21.918 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T19:00:22.000 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T19:00:22.022 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T19:00:22.043 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T19:00:22.122 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T19:00:22.130 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:00:22.280 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T19:00:22.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:00:22.340 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:00:22.347 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T19:00:22.465 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:00:22.497 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:00:22.842 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T19:00:22.915 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T19:00:22.944 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:00:23.122 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:00:23.162 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:00:23.256 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:00:23.288 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:00:23.327 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T19:00:23.355 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:00:23.387 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T19:00:23.421 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T19:00:23.546 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T19:00:23.609 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T19:00:23.651 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:00:23.691 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:00:23.722 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T19:00:23.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T19:00:23.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T19:00:23.997 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T19:00:24.021 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T19:00:24.027 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T19:00:24.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T19:00:24.133 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T19:00:24.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T19:00:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T19:00:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T19:00:24.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T19:00:24.369 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T19:00:24.483 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T19:00:24.548 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T19:00:24.616 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T19:00:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T19:00:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T19:00:24.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T19:00:24.895 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T19:00:24.931 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T19:00:24.967 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T19:00:25.049 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T19:00:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T19:00:25.126 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T19:00:25.217 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T19:00:25.400 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T19:00:25.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T19:00:26.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T19:00:26.277 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T19:00:26.312 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T19:00:26.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T19:00:26.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T19:00:26.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T19:00:26.673 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T19:00:26.740 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T19:00:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T19:00:26.832 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T19:00:26.849 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T19:00:26.901 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T19:00:26.927 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T19:00:26.965 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T19:00:26.978 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T19:00:27.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T19:00:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T19:00:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T19:00:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T19:00:27.239 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T19:00:27.383 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T19:00:27.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T19:00:27.454 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:00:27.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T19:00:27.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T19:00:27.612 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:00:27.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T19:00:27.703 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T19:00:27.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T19:00:27.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T19:00:27.820 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T19:00:27.853 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T19:00:27.886 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T19:00:28.072 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T19:00:28.250 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T19:00:28.258 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T19:00:28.290 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:00:28.293 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T19:00:28.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T19:00:28.378 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T19:00:28.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T19:00:28.457 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:00:28.483 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T19:00:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T19:00:28.529 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T19:00:28.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T19:00:28.625 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T19:00:28.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T19:00:28.747 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T19:00:28.888 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T19:00:28.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T19:00:28.997 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T19:00:29.067 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T19:00:29.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T19:00:29.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T19:00:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T19:00:30.318 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T19:00:30.359 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T19:00:30.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T19:00:30.555 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T19:00:30.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T19:00:30.702 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T19:00:30.928 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T19:00:30.958 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T19:00:31.059 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T19:00:31.495 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:31.762 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:31.894 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T19:00:32.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T19:00:32.467 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:32.510 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:00:32.664 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:00:32.664 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T19:00:32.664 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T19:00:32.664 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:32.753 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:00:33.637 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T19:00:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T19:00:33.772 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T19:00:33.814 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T19:00:34.223 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T19:00:34.267 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:00:34.421 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:00:34.458 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T19:00:34.516 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T19:00:35.087 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T19:00:35.112 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:00:35.203 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:00:35.231 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T19:00:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:38.641 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:00:38.680 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:00:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:00:38.835 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T19:00:38.835 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T19:00:38.835 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:38.890 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-04-25T19:00:46.104 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-04-25T19:00:46.105 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-04-25T19:00:46.105 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:46.381 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:00:46.429 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:00:46.430 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:46.430 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T19:00:46.430 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:48.194 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:00:48.272 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:00:48.272 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:48.272 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T19:00:48.272 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:00:48.272 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:00:48.273 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:48.531 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:00:48.598 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:00:48.598 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:48.598 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T19:00:48.599 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:00:48.599 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:00:48.599 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:49.106 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:00:49.175 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:00:49.176 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:49.176 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T19:00:49.176 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:00:49.176 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:00:49.176 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:49.307 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T19:00:49.367 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T19:00:49.446 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T19:00:49.679 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T19:00:49.744 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T19:00:49.831 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T19:00:49.940 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:00:50.110 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:00:50.179 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T19:00:50.218 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T19:00:50.290 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T19:00:50.960 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:00:50.996 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:00:51.086 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T19:00:51.199 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T19:00:51.230 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:00:52.062 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-25T19:00:52.063 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:52.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:00:52.321 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:00:52.321 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:52.321 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T19:00:52.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:00:52.591 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:00:53.697 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:00:53.724 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:00:53.872 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:00:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:00:54.052 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T19:00:54.084 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:00:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:00:54.176 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:54.243 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:00:54.282 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:00:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:00:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:54.456 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T19:00:54.531 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T19:00:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:00:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:00:55.033 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:55.033 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T19:00:55.033 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:00:55.034 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:00:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:00:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T19:00:55.226 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T19:00:55.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T19:00:55.542 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T19:00:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T19:00:55.587 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:00:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:00:55.662 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:55.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T19:00:55.713 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:00:55.783 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:00:55.783 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:55.783 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T19:00:55.783 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:55.804 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:00:55.856 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:00:55.931 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:00:55.931 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:55.931 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T19:00:55.931 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:00:55.932 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:00:55.932 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:00:56.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T19:00:56.084 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T19:00:56.124 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T19:00:56.323 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:00:56.392 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:00:56.393 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:56.393 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T19:00:56.393 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:00:56.393 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:00:56.393 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:00:56.823 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:00:56.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:00:56.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T19:00:57.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T19:00:57.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:00:58.357 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:00:58.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:00:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:00:59.514 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:00:59.661 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:00:59.842 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T19:00:59.875 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:00:59.961 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:00:59.962 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:00:59.962 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T19:00:59.962 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:00:59.962 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:00:59.962 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:00.017 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:01:00.055 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:01:00.231 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T19:01:00.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T19:01:01.162 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T19:01:01.217 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T19:01:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T19:01:01.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T19:01:01.348 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T19:01:01.361 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T19:01:01.436 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:01:01.436 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:01:01.437 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T19:01:01.437 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:01:01.437 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:01:01.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:01.520 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:01:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:01:01.584 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:01:01.584 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T19:01:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:01.663 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:01:01.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:01.770 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T19:01:01.833 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T19:01:01.861 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T19:01:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:01:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:02.766 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:02.817 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:02.869 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:04.229 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T19:01:04.343 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T19:01:04.433 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T19:01:04.621 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T19:01:04.666 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T19:01:04.666 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:01:04.848 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:01:04.888 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:01:05.090 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:01:05.219 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:05.510 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:05.511 INFO:teuthology.orchestra.run.smithi146.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T19:01:05.511 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:06.903 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T19:01:06.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T19:01:07.028 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T19:01:07.071 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T19:01:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T19:01:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T19:01:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T19:01:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T19:01:08.417 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:08.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:08.520 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:01:09.869 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T19:01:09.980 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T19:01:10.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T19:01:10.246 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T19:01:10.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T19:01:10.286 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:01:10.421 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:01:10.468 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:01:10.468 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:01:10.592 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:01:10.685 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:10.978 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:10.978 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T19:01:10.978 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:01:11.483 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T19:01:11.484 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T19:01:11.485 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T19:01:11.485 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T19:01:11.496 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T19:01:11.496 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T19:01:11.496 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T19:01:11.496 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T19:01:11.497 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T19:01:11.498 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T19:01:11.499 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T19:01:11.500 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T19:01:11.501 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T19:01:11.502 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T19:01:11.503 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T19:01:11.504 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:01:11.505 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:13.925 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.926 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.927 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T19:01:13.928 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T19:01:13.929 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T19:01:13.930 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T19:01:13.931 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T19:01:13.932 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T19:01:13.933 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.934 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T19:01:13.935 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T19:01:13.936 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T19:01:13.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:13.937 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T19:01:14.537 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T19:01:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T19:01:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T19:01:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T19:01:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T19:01:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T19:01:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T19:01:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T19:01:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T19:01:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T19:01:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:01:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T19:01:19.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T19:01:19.135 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T19:01:19.137 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.138 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T19:01:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T19:01:19.140 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T19:01:19.141 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T19:01:19.142 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T19:01:19.143 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T19:01:19.144 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T19:01:19.145 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:19.146 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T19:01:19.792 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:19.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:01:20.015 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T19:01:20.070 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T19:01:20.070 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T19:01:20.071 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T19:01:20.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:01:20.214 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T19:01:20.270 INFO:teuthology.orchestra.run.smithi146.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T19:01:20.270 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T19:01:20.270 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T19:01:20.272 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T19:01:20.272 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:20.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T19:01:20.352 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:20.352 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T19:01:20.420 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T19:01:20.420 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:20.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T19:01:20.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T19:01:20.587 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:20.588 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T19:01:20.650 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T19:01:20.749 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T19:01:20.749 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:20.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T19:01:20.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T19:01:20.924 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:20.924 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T19:01:20.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T19:01:21.096 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T19:01:21.097 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:21.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T19:01:21.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T19:01:21.270 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:21.270 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T19:01:21.343 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T19:01:21.451 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T19:01:21.537 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T19:01:21.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-04-25T19:01:21.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-04-25T19:01:21.619 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T19:01:21.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T19:01:21.625 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T19:01:21.706 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T19:01:21.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T19:01:21.711 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T19:01:21.801 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T19:01:21.801 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T19:01:21.801 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T19:01:21.801 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:21.801 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T19:01:21.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:22.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:22.091 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:22.091 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T19:01:22.147 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:22.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:22.445 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T19:01:22.445 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T19:01:22.445 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673292', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPt4Wyv06Qc5jTkQ+FBy0+awodsrXtLqkUijs/YZAajDL0HWBWcftf8CvA5gDoQfxUUc2kOdNyPhoD78py8gEc=', 'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAWUdEMFgpSjFaq2rLRmc1QqrNOnczHC+/5qujm7oPiKXCCIykkFxtjcKdxGGF83TDxu26e7aeh6KxDNUxhRSPw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T19:01:22.446 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T19:01:22.450 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T19:01:22.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T19:01:22.527 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T19:01:22.608 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:22.608 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T19:01:22.646 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T19:01:22.646 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 19:01:11.422873191 +0000 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 18:58:50.047661677 +0000 2024-04-25T19:01:22.726 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 18:58:50.047661677 +0000 2024-04-25T19:01:22.727 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 18:58:50.047661677 +0000 2024-04-25T19:01:22.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T19:01:22.825 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T19:01:22.825 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T19:01:22.825 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000323383 s, 1.6 MB/s 2024-04-25T19:01:22.828 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T19:01:22.866 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-25T19:01:22.946 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 19:01:11.422873191 +0000 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 18:58:50.639647912 +0000 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 18:58:50.639647912 +0000 2024-04-25T19:01:22.947 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 18:58:50.639647912 +0000 2024-04-25T19:01:22.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T19:01:23.046 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T19:01:23.046 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T19:01:23.046 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00029585 s, 1.7 MB/s 2024-04-25T19:01:23.050 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T19:01:23.091 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 19:01:11.423873168 +0000 2024-04-25T19:01:23.181 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 18:58:51.246633798 +0000 2024-04-25T19:01:23.182 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 18:58:51.246633798 +0000 2024-04-25T19:01:23.182 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 18:58:51.246633798 +0000 2024-04-25T19:01:23.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T19:01:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T19:01:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T19:01:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000287541 s, 1.8 MB/s 2024-04-25T19:01:23.247 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T19:01:23.288 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-25T19:01:23.366 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 19:01:11.424873145 +0000 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 18:58:51.889618846 +0000 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 18:58:51.889618846 +0000 2024-04-25T19:01:23.367 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 18:58:51.889618846 +0000 2024-04-25T19:01:23.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T19:01:23.466 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-25T19:01:23.466 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-25T19:01:23.466 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000371672 s, 1.4 MB/s 2024-04-25T19:01:23.469 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T19:01:23.515 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:23.515 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:23.515 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T19:01:23.570 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T19:01:23.570 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-04-25T19:01:23.618 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T19:01:23.618 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:23.618 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 19:01:06.000922721 +0000 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 18:58:50.034220961 +0000 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 18:58:50.034220961 +0000 2024-04-25T19:01:23.619 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 18:58:50.034220961 +0000 2024-04-25T19:01:23.619 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T19:01:23.695 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T19:01:23.695 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T19:01:23.695 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000459827 s, 1.1 MB/s 2024-04-25T19:01:23.698 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T19:01:23.755 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-04-25T19:01:23.800 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T19:01:23.800 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 19:01:06.001922706 +0000 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 18:58:50.660211159 +0000 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 18:58:50.660211159 +0000 2024-04-25T19:01:23.801 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 18:58:50.660211159 +0000 2024-04-25T19:01:23.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000328218 s, 1.6 MB/s 2024-04-25T19:01:23.926 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T19:01:23.974 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-04-25T19:01:24.012 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T19:01:24.012 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:24.012 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-25T19:01:24.012 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:24.013 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:24.013 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 19:01:06.003922677 +0000 2024-04-25T19:01:24.013 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 18:58:51.254201858 +0000 2024-04-25T19:01:24.013 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 18:58:51.254201858 +0000 2024-04-25T19:01:24.013 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 18:58:51.254201858 +0000 2024-04-25T19:01:24.013 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T19:01:24.136 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T19:01:24.136 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T19:01:24.136 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000359115 s, 1.4 MB/s 2024-04-25T19:01:24.139 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T19:01:24.189 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 19:01:06.003922677 +0000 2024-04-25T19:01:24.231 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 18:58:51.844192619 +0000 2024-04-25T19:01:24.232 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 18:58:51.844192619 +0000 2024-04-25T19:01:24.232 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 18:58:51.844192619 +0000 2024-04-25T19:01:24.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T19:01:24.364 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T19:01:24.364 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T19:01:24.364 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000341644 s, 1.5 MB/s 2024-04-25T19:01:24.367 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T19:01:24.413 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:24.413 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T19:01:24.413 INFO:tasks.ceph:Generating config... 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] pid file = 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T19:01:24.415 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T19:01:24.415 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T19:01:24.415 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T19:01:24.415 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T19:01:24.415 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T19:01:24.415 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T19:01:24.415 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T19:01:24.416 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T19:01:24.416 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T19:01:24.416 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T19:01:24.416 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T19:01:24.417 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T19:01:24.417 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T19:01:24.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T19:01:24.579 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T19:01:24.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T19:01:24.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:01:24.825 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.123'), ('mon.b', '172.21.15.146'), ('mon.c', '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]')] 2024-04-25T19:01:24.826 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.123,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T19:01:24.827 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:24.828 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T19:01:24.865 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.123 --add b 172.21.15.146 --addv c '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: generated fsid a2d72801-e932-4284-9776-fce727cd5af1 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 0 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:fsid a2d72801-e932-4284-9776-fce727cd5af1 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:last_changed 2024-04-25T19:01:25.058398+0000 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-25T19:01:25.058398+0000 2024-04-25T19:01:25.059 INFO:teuthology.orchestra.run.smithi123.stdout:min_mon_release 17 (quincy) 2024-04-25T19:01:25.060 INFO:teuthology.orchestra.run.smithi123.stdout:election_strategy: 1 2024-04-25T19:01:25.060 INFO:teuthology.orchestra.run.smithi123.stdout:0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-25T19:01:25.060 INFO:teuthology.orchestra.run.smithi123.stdout:1: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-04-25T19:01:25.060 INFO:teuthology.orchestra.run.smithi123.stdout:2: [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] mon.c 2024-04-25T19:01:25.060 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T19:01:25.068 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T19:01:25.107 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a2d72801-e932-4284-9776-fce727cd5af1... 2024-04-25T19:01:25.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T19:01:25.153 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.256 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-04-25T19:01:25.257 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-04-25T19:01:25.258 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on skipped message = False 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = a2d72801-e932-4284-9776-fce727cd5af1 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-04-25T19:01:25.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-04-25T19:01:25.260 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-04-25T19:01:25.261 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T19:01:25.262 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T19:01:25.263 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: mon down mkfs grace = 300 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-04-25T19:01:25.264 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-25T19:01:25.265 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-04-25T19:01:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-25T19:01:25.267 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-25T19:01:25.267 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-25T19:01:25.294 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-04-25T19:01:25.295 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-04-25T19:01:25.295 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T19:01:25.295 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-04-25T19:01:25.295 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.295 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-04-25T19:01:25.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.297 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-04-25T19:01:25.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-04-25T19:01:25.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-04-25T19:01:25.301 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.123,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on skipped message = False 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = a2d72801-e932-4284-9776-fce727cd5af1 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-04-25T19:01:25.302 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-04-25T19:01:25.303 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.304 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.305 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-04-25T19:01:25.306 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-04-25T19:01:25.307 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:01:25.308 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-04-25T19:01:25.309 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.310 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-04-25T19:01:25.311 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-04-25T19:01:25.312 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-04-25T19:01:25.313 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-04-25T19:01:25.313 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-04-25T19:01:25.313 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-04-25T19:01:25.313 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-04-25T19:01:25.326 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T19:01:25.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T19:01:25.503 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T19:01:25.503 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:25.503 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T19:01:25.543 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:25.543 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T19:01:25.620 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:01:25.620 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:25.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T19:01:25.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:01:25.747 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:01:25.747 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:25.837 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:01:25.838 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:25.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T19:01:25.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:01:25.951 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:01:25.951 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:26.045 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T19:01:26.045 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T19:01:26.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T19:01:26.236 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T19:01:26.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T19:01:26.483 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T19:01:26.500 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T19:01:26.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T19:01:26.704 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T19:01:26.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T19:01:26.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T19:01:26.995 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T19:01:27.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T19:01:27.085 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T19:01:27.308 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T19:01:27.323 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T19:01:27.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T19:01:27.629 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T19:01:27.645 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T19:01:27.720 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T19:01:27.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T19:01:27.909 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T19:01:27.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T19:01:28.146 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T19:01:28.194 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T19:01:28.385 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T19:01:28.433 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T19:01:28.616 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T19:01:28.660 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T19:01:28.660 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T19:01:28.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T19:01:28.749 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:28.750 INFO:tasks.ceph:role: osd.0 2024-04-25T19:01:28.750 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:01:28.750 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:28.829 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:29.241 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T19:01:29.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T19:01:29.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T19:01:29.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T19:01:29.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T19:01:29.555 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:29.556 INFO:tasks.ceph:role: osd.1 2024-04-25T19:01:29.556 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:01:29.556 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T19:01:29.636 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:29.636 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:29.637 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:30.023 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T19:01:30.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T19:01:30.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T19:01:30.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T19:01:30.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T19:01:30.290 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:30.290 INFO:tasks.ceph:role: osd.2 2024-04-25T19:01:30.290 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:01:30.290 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T19:01:30.374 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:30.374 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:30.374 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:30.375 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:30.376 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:30.805 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T19:01:30.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T19:01:30.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T19:01:30.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T19:01:31.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T19:01:31.127 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:31.127 INFO:tasks.ceph:role: osd.3 2024-04-25T19:01:31.128 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:01:31.128 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:31.215 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:31.216 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:31.626 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-25T19:01:31.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-25T19:01:31.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T19:01:31.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T19:01:31.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:32.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.028+0000 7f8c2c20d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T19:01:32.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.028+0000 7f8c2c20d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T19:01:32.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.028+0000 7f8c2c20d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:01:32.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.028+0000 7f8c2c20d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:01:32.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.028+0000 7f8c2c20d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:01:32.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:32.032+0000 7f8c2c20d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T19:01:35.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T19:01:35.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:35.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.277+0000 7f970de0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T19:01:35.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.278+0000 7f970de0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T19:01:35.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.278+0000 7f970de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:01:35.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.278+0000 7f970de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:01:35.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.278+0000 7f970de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:01:35.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:35.280+0000 7f970de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T19:01:38.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T19:01:38.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:38.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.526+0000 7febfa9ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T19:01:38.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.527+0000 7febfa9ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T19:01:38.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.527+0000 7febfa9ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:01:38.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.527+0000 7febfa9ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:01:38.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.527+0000 7febfa9ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:01:38.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:38.529+0000 7febfa9ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T19:01:41.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T19:01:41.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:41.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.796+0000 7fe452c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T19:01:41.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.797+0000 7fe452c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T19:01:41.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.797+0000 7fe452c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:01:41.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.797+0000 7fe452c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:01:41.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.797+0000 7fe452c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:01:41.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:01:41.799+0000 7fe452c15640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T19:01:44.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T19:01:44.861 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T19:01:44.950 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:44.951 INFO:tasks.ceph:role: osd.4 2024-04-25T19:01:44.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:01:44.951 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T19:01:45.042 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:45.043 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:45.443 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T19:01:45.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T19:01:45.450 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T19:01:45.572 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T19:01:45.652 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T19:01:45.728 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:45.729 INFO:tasks.ceph:role: osd.5 2024-04-25T19:01:45.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:01:45.729 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T19:01:45.802 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:45.802 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:45.802 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:45.802 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:45.802 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:45.803 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:45.803 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:45.803 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:45.803 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:45.803 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:46.157 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T19:01:46.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T19:01:46.163 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T19:01:46.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T19:01:46.330 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T19:01:46.396 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:46.396 INFO:tasks.ceph:role: osd.6 2024-04-25T19:01:46.396 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:01:46.396 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:46.472 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:46.857 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T19:01:46.862 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T19:01:46.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T19:01:46.973 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T19:01:47.054 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T19:01:47.123 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:01:47.124 INFO:tasks.ceph:role: osd.7 2024-04-25T19:01:47.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:01:47.124 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:01:47.204 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:01:47.205 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:01:47.205 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:01:47.205 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:01:47.205 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:01:47.629 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T19:01:47.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T19:01:47.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T19:01:47.780 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T19:01:47.864 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:48.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.070+0000 7fe704db1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T19:01:48.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.071+0000 7fe704db1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T19:01:48.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.071+0000 7fe704db1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:01:48.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.071+0000 7fe704db1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:01:48.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.071+0000 7fe704db1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:01:48.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:48.074+0000 7fe704db1640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T19:01:51.286 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T19:01:51.368 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:51.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.564+0000 7fcb3f133640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T19:01:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.565+0000 7fcb3f133640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T19:01:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.565+0000 7fcb3f133640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:01:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.565+0000 7fcb3f133640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:01:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.565+0000 7fcb3f133640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:01:51.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:51.567+0000 7fcb3f133640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T19:01:52.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:52.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:01:54.782 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T19:01:54.861 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:55.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.054+0000 7fec0f1a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T19:01:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.055+0000 7fec0f1a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T19:01:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.055+0000 7fec0f1a5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:01:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.055+0000 7fec0f1a5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:01:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.055+0000 7fec0f1a5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:01:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:55.057+0000 7fec0f1a5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T19:01:58.085 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T19:01:58.171 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:01:58.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.374+0000 7f6391e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T19:01:58.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.374+0000 7f6391e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T19:01:58.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.374+0000 7f6391e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:01:58.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.374+0000 7f6391e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:01:58.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.374+0000 7f6391e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:01:58.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:01:58.377+0000 7f6391e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T19:02:01.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T19:02:01.687 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T19:02:01.687 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:01.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T19:02:01.778 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:01.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T19:02:01.852 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:01.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T19:02:01.981 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:01.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T19:02:02.067 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:02.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T19:02:02.156 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:02.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T19:02:02.280 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:02.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T19:02:02.361 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T19:02:02.452 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.452 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T19:02:02.535 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T19:02:02.615 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.615 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T19:02:02.699 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T19:02:02.784 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.784 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T19:02:02.867 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:02.867 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T19:02:02.950 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:02.950 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T19:02:03.008 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:03.008 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T19:02:03.059 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:03.059 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T19:02:03.115 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:03.115 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T19:02:03.169 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T19:02:03.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T19:02:03.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T19:02:03.254 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-04-25T19:02:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGqCpmUYgMDhAA5gq7FeuAAoyFNZuiFpRDRQ== 2024-04-25T19:02:03.254 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGqCpmBVn2KRAAWo2mVIL6zYcxXNeUHws5EQ== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGqCpmFt5KOxAAnXURk+vegj7PL5IzJYdqfQ== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCMqCpmMne7ARAAD/fsufO2UPnMG5vHj5+/Kg== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPqCpmBzCcEBAAxG14/PkfSEXERSIOZ4SOFg== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCSqCpmghBrHxAAzjHRr0EPcDdh0uJ9B0ECEw== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCVqCpmeSyOLxAAvABp5eQNxeOenxSPz4ithA== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCGqCpmS1zAHBAAal0LOYhhk4JQSXYcWs58kw== 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-25T19:02:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHqCpm4OhZEhAAN00cjmzmkk8s36+QKSljUw== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHqCpmCfiAJRAAYE9F8O1uISrXo1UJk5eZjw== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCcqCpmWgs9BBAApfP4lek/yUB29DLCEBPJIQ== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCfqCpmqvSsIRAA0U6IDjWS2+lzkqcIFEaYIA== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCjqCpmUUxEAxAABLXfQutj7ad6CT44UGnW7A== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCmqCpmQF5TFhAAxICQfrhe5g/wOs0xChEM6g== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHqCpmMqUvNhAA5Rf0ooG9ZwRKfPr3VFwX6A== 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2024-04-25T19:02:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCIqCpmt3C3CBAAF5ln+HFzFpzt2nVjrZDSsQ== 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCIqCpmhpLrFhAAOn+ZCacMA0yVBegykPu+OQ== 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCIqCpmGpSvJBAAiRnXzCJvTv4D6wPmEDOeEw== 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2024-04-25T19:02:03.257 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGqCpmUYgMDhAA5gq7FeuAAoyFNZuiFpRDRQ== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGqCpmBVn2KRAAWo2mVIL6zYcxXNeUHws5EQ== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGqCpmFt5KOxAAnXURk+vegj7PL5IzJYdqfQ== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCMqCpmMne7ARAAD/fsufO2UPnMG5vHj5+/Kg== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCPqCpmBzCcEBAAxG14/PkfSEXERSIOZ4SOFg== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCSqCpmghBrHxAAzjHRr0EPcDdh0uJ9B0ECEw== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCVqCpmeSyOLxAAvABp5eQNxeOenxSPz4ithA== 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-04-25T19:02:03.258 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGqCpmS1zAHBAAal0LOYhhk4JQSXYcWs58kw== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHqCpm4OhZEhAAN00cjmzmkk8s36+QKSljUw== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHqCpmCfiAJRAAYE9F8O1uISrXo1UJk5eZjw== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCcqCpmWgs9BBAApfP4lek/yUB29DLCEBPJIQ== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCfqCpmqvSsIRAA0U6IDjWS2+lzkqcIFEaYIA== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCjqCpmUUxEAxAABLXfQutj7ad6CT44UGnW7A== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCmqCpmQF5TFhAAxICQfrhe5g/wOs0xChEM6g== 2024-04-25T19:02:03.259 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHqCpmMqUvNhAA5Rf0ooG9ZwRKfPr3VFwX6A== 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCIqCpmt3C3CBAAF5ln+HFzFpzt2nVjrZDSsQ== 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCIqCpmhpLrFhAAOn+ZCacMA0yVBegykPu+OQ== 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2024-04-25T19:02:03.260 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCIqCpmGpSvJBAAiRnXzCJvTv4D6wPmEDOeEw== 2024-04-25T19:02:03.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:02:03.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:02:03.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:03.527 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:03.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:03.742 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:03.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:03.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.173 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.390 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.600 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:04.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:02:04.815 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:02:05.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:05.026 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:05.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:05.243 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:02:05.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:05.458 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:05.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:05.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:05.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:05.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:06.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:06.099 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:02:06.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.302 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.728 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:06.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:02:07.163 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T19:02:07.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T19:02:07.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:02:07.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T19:02:07.563 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T19:02:07.655 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:02:07.889 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T19:02:07.958 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T19:02:08.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:02:08.262 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T19:02:08.326 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:02:08.334 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:02:08.382 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T19:02:08.383 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T19:02:08.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T19:02:08.388 INFO:tasks.ceph.mon.a:Started 2024-04-25T19:02:08.388 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T19:02:08.388 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T19:02:08.429 INFO:tasks.ceph.mon.b:Started 2024-04-25T19:02:08.429 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T19:02:08.430 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T19:02:08.437 INFO:tasks.ceph.mon.c:Started 2024-04-25T19:02:08.437 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T19:02:08.437 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T19:02:08.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T19:02:08.443 INFO:tasks.ceph.mgr.y:Started 2024-04-25T19:02:08.443 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T19:02:08.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T19:02:08.448 INFO:tasks.ceph.mgr.x:Started 2024-04-25T19:02:08.448 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T19:02:08.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T19:02:08.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.689+0000 7fe7ff800640 1 Processor -- start 2024-04-25T19:02:08.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.689+0000 7fe7ff800640 1 -- start start 2024-04-25T19:02:08.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.691+0000 7fe7ff800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.691+0000 7fe7ff800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.691+0000 7fe7ff800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f80d48b0 con 0x7fe7f80d13a0 2024-04-25T19:02:08.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.691+0000 7fe7ff800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80d54c0 con 0x7fe7f815f760 2024-04-25T19:02:08.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.691+0000 7fe7ff800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80d4b90 con 0x7fe7f80d0ed0 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7efe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fce00640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe7f80d13a0 0x7fe7f80d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:49694/0 (socket says 172.21.15.123:49694) 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fce00640 1 -- 172.21.15.123:0/918134856 learned_addr learned my addr 172.21.15.123:0/918134856 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.692+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.693+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:08.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.693+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.894+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.894+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:02:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:08.895+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:09.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.296+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.297+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.297+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.297+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:09.297+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:02:10.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.098+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:10.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.098+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:10.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.098+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:10.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.098+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:10.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.098+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:02:10.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:10.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.099+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:02:10.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.099+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:10.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:10.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:10.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 msgr2=0x7fe7f80b4ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7f80d0ed0 0x7fe7f80b4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 msgr2=0x7fe7f815fb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.123:3300/0 conn(0x7fe7f815f760 0x7fe7f815fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 -- 172.21.15.123:0/918134856 >> v1:172.21.15.146:6789/0 conn(0x7fe7f80d13a0 legacy=0x7fe7f80d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 -- 172.21.15.123:0/918134856 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f80d4e70 con 0x7fe7d4003680 2024-04-25T19:02:11.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.692+0000 7fe7f6a00640 1 -- 172.21.15.123:0/918134856 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80d54c0 con 0x7fe7d4007380 2024-04-25T19:02:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.693+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.693+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.693+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.693+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.693+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:11.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.894+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.895+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:11.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.895+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:11.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.895+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:11.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:11.895+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:12.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:12.296+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:12.297+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:12.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:12.297+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:12.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:12.297+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:12.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:12.297+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:02:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.099+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:02:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.099+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:02:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:02:13.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.099+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:02:13.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.950+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335161564 0 0) 0x7fe7f80d4e70 con 0x7fe7d4003680 2024-04-25T19:02:13.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.950+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7dc003520 con 0x7fe7d4003680 2024-04-25T19:02:13.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 490842007 0 0) 0x7fe7dc003520 con 0x7fe7d4003680 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 msgr2=0x7fe7d4009890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 --2- 172.21.15.123:0/918134856 >> v2:172.21.15.146:3300/0 conn(0x7fe7d4007380 0x7fe7d4009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d400b050 con 0x7fe7d4003680 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fe7e80016f0 con 0x7fe7d4003680 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 >> v1:172.21.15.123:6789/0 conn(0x7fe7d4003680 legacy=0x7fe7d4005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7fd800640 1 -- 172.21.15.123:0/918134856 reap_dead start 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.954+0000 7fe7f7e00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.955+0000 7fe7f7e00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.955+0000 7fe7f7e00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7dc00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.955+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d54c0 con 0x7fe7dc004800 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.955+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00c320 con 0x7fe7dc005380 2024-04-25T19:02:13.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.955+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7dc00c520 con 0x7fe7dc0039b0 2024-04-25T19:02:13.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7fce00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7fd800640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7dc00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 msgr2=0x7fe7dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 msgr2=0x7fe7dc00ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.956+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7dc00ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.957+0000 7fe7efe00640 1 -- 172.21.15.123:0/918134856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc003520 con 0x7fe7dc004800 2024-04-25T19:02:13.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.959+0000 7fe7efe00640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7dc004c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0019cc0 tx=0x7fe7dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cb033b635a2199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:13.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.963+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0032070 con 0x7fe7dc004800 2024-04-25T19:02:13.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.963+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e002cd00 con 0x7fe7dc004800 2024-04-25T19:02:13.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.963+0000 7fe7f7e00640 1 -- 172.21.15.123:0/918134856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0019940 con 0x7fe7dc004800 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 -- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 msgr2=0x7fe7dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7dc004c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0019cc0 tx=0x7fe7dc005830 comp rx=0 tx=0).stop 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 -- 172.21.15.123:0/918134856 shutdown_connections 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7dc00ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7dc004c10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 --2- 172.21.15.123:0/918134856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7dc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 -- 172.21.15.123:0/918134856 >> 172.21.15.123:0/918134856 conn(0x7fe7f80d6500 msgr2=0x7fe7f80b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 -- 172.21.15.123:0/918134856 shutdown_connections 2024-04-25T19:02:13.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.965+0000 7fe7ff800640 1 -- 172.21.15.123:0/918134856 wait complete. 2024-04-25T19:02:13.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.968+0000 7fe7ff800640 1 Processor -- start 2024-04-25T19:02:13.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.969+0000 7fe7ff800640 1 -- start start 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7f8158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7f80ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d45c0 con 0x7fe7dc004800 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d4440 con 0x7fe7dc005380 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7ff800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80b4ee0 con 0x7fe7dc0039b0 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33926/0 (socket says 172.21.15.123:33926) 2024-04-25T19:02:13.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7efe00640 1 -- 172.21.15.123:0/279854087 learned_addr learned my addr 172.21.15.123:0/279854087 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7fce00640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7f8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.971+0000 7fe7fd800640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7f80ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 -- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 msgr2=0x7fe7f8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7f8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 -- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 msgr2=0x7fe7f80ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7f80ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 -- 172.21.15.123:0/279854087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f8158d80 con 0x7fe7dc004800 2024-04-25T19:02:13.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7efe00640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0019de0 tx=0x7fe7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:13.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.972+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0044020 con 0x7fe7dc004800 2024-04-25T19:02:13.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e0003610 con 0x7fe7dc004800 2024-04-25T19:02:13.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f80cae80 con 0x7fe7dc004800 2024-04-25T19:02:13.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0003a00 con 0x7fe7dc004800 2024-04-25T19:02:13.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f80cb440 con 0x7fe7dc004800 2024-04-25T19:02:13.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7e0047020 con 0x7fe7dc004800 2024-04-25T19:02:13.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.973+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7e003fac0 con 0x7fe7dc004800 2024-04-25T19:02:13.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.974+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f8076a40 con 0x7fe7dc004800 2024-04-25T19:02:13.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:13.978+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe7e004e030 con 0x7fe7dc004800 2024-04-25T19:02:14.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.053+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe7f80d3f60 con 0x7fe7dc004800 2024-04-25T19:02:14.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.059+0000 7fe7f5600640 1 -- 172.21.15.123:0/279854087 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fe7e003fd10 con 0x7fe7dc004800 2024-04-25T19:02:14.061 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-04-25T19:02:14.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 msgr2=0x7fe7f8156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:14.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0019de0 tx=0x7fe7e00046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 shutdown_connections 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc005380 0x7fe7f80ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7dc004800 0x7fe7f8156ae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 --2- 172.21.15.123:0/279854087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7dc0039b0 0x7fe7f8158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.060+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 >> 172.21.15.123:0/279854087 conn(0x7fe7f80d6500 msgr2=0x7fe7f80d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.061+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 shutdown_connections 2024-04-25T19:02:14.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:14.061+0000 7fe7ff800640 1 -- 172.21.15.123:0/279854087 wait complete. 2024-04-25T19:02:14.088 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T19:02:14.088 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T19:02:14.088 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:14.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T19:02:14.117 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:14.115+0000 7f20e2a35040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:02:14.128 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:14.126+0000 7f359bc2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:02:14.158 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:14.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T19:02:14.276 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:14.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T19:02:14.305 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:14.303+0000 7f20e2a35040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:14.314 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:14.312+0000 7f359bc2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:14.369 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:14.368+0000 7f20e2a35040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:02:14.374 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:14.372+0000 7f359bc2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:02:14.382 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:14.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T19:02:14.497 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:14.497 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T19:02:14.573 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:14.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T19:02:14.658 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:14.656+0000 7f20e2a35040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:02:14.682 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:14.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T19:02:14.696 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:14.694+0000 7f359bc2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:02:14.710 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:14.709+0000 7f20e2a35040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:02:14.753 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:14.751+0000 7f359bc2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:02:14.775 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:02:14.775 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T19:02:14.891 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 45334943-7f6e-4b40-8809-22b648c39b7d 0 2024-04-25T19:02:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.183+0000 7ffbf3200640 1 Processor -- start 2024-04-25T19:02:15.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.184+0000 7ffbf3200640 1 -- start start 2024-04-25T19:02:15.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.185+0000 7ffbf3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbec0b4c10 0x7ffbec160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.185+0000 7ffbf3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbf3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbec0b9600 con 0x7ffbec06d630 2024-04-25T19:02:15.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbf3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffbec06cc60 con 0x7ffbec0b4c10 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbf3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffbec0d4f20 con 0x7ffbec0b91f0 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:46792/0 (socket says 172.21.15.146:46792) 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 -- 172.21.15.146:0/3884137132 learned_addr learned my addr 172.21.15.146:0/3884137132 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbebe00640 1 --2- 172.21.15.146:0/3884137132 >> v2:172.21.15.123:3300/0 conn(0x7ffbec0b4c10 0x7ffbec160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 -- 172.21.15.146:0/3884137132 >> v2:172.21.15.123:3300/0 conn(0x7ffbec0b4c10 msgr2=0x7ffbec160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 --2- 172.21.15.146:0/3884137132 >> v2:172.21.15.123:3300/0 conn(0x7ffbec0b4c10 0x7ffbec160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 -- 172.21.15.146:0/3884137132 >> v1:172.21.15.123:6789/0 conn(0x7ffbec06d630 legacy=0x7ffbec163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.186+0000 7ffbe3e00640 1 -- 172.21.15.146:0/3884137132 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec0d5200 con 0x7ffbec0b91f0 2024-04-25T19:02:15.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.187+0000 7ffbe3e00640 1 --2- 172.21.15.146:0/3884137132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009870 tx=0x7ffbd402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c474ba2a874cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.188+0000 7ffbeb400640 1 -- 172.21.15.146:0/3884137132 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4037070 con 0x7ffbec0b91f0 2024-04-25T19:02:15.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.188+0000 7ffbeb400640 1 -- 172.21.15.146:0/3884137132 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd4032440 con 0x7ffbec0b91f0 2024-04-25T19:02:15.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.188+0000 7ffbeb400640 1 -- 172.21.15.146:0/3884137132 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4032810 con 0x7ffbec0b91f0 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.188+0000 7ffbf3200640 1 -- 172.21.15.146:0/3884137132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 msgr2=0x7ffbec06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.188+0000 7ffbf3200640 1 --2- 172.21.15.146:0/3884137132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009870 tx=0x7ffbd402a250 comp rx=0 tx=0).stop 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 -- 172.21.15.146:0/3884137132 shutdown_connections 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 --2- 172.21.15.146:0/3884137132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec06d0f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 --2- 172.21.15.146:0/3884137132 >> v2:172.21.15.123:3300/0 conn(0x7ffbec0b4c10 0x7ffbec160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 -- 172.21.15.146:0/3884137132 >> 172.21.15.146:0/3884137132 conn(0x7ffbec0b7690 msgr2=0x7ffbec0b7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 -- 172.21.15.146:0/3884137132 shutdown_connections 2024-04-25T19:02:15.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.189+0000 7ffbf3200640 1 -- 172.21.15.146:0/3884137132 wait complete. 2024-04-25T19:02:15.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.191+0000 7ffbf3200640 1 Processor -- start 2024-04-25T19:02:15.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.191+0000 7ffbf3200640 1 -- start start 2024-04-25T19:02:15.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.192+0000 7ffbf3200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.192+0000 7ffbf3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 0x7ffbec1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec06c8f0 con 0x7ffbec0b4c10 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf3200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec06c610 con 0x7ffbec06d630 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffbec06da40 con 0x7ffbec0b91f0 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf1200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52938/0 (socket says 172.21.15.146:52938) 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 -- 172.21.15.146:0/1638301884 learned_addr learned my addr 172.21.15.146:0/1638301884 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbe3e00640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 0x7ffbec1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 -- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 msgr2=0x7ffbec159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 -- 172.21.15.146:0/1638301884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 msgr2=0x7ffbec1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 0x7ffbec1578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbebe00640 1 -- 172.21.15.146:0/1638301884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec158b40 con 0x7ffbec06d630 2024-04-25T19:02:15.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbe3e00640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 0x7ffbec1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:15.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.193+0000 7ffbf1200640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbebe00640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00e870 tx=0x7ffbdc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc017070 con 0x7ffbec06d630 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00c940 con 0x7ffbec06d630 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec158e20 con 0x7ffbec06d630 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00cd30 con 0x7ffbec06d630 2024-04-25T19:02:15.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec1593e0 con 0x7ffbec06d630 2024-04-25T19:02:15.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffbdc01a070 con 0x7ffbec06d630 2024-04-25T19:02:15.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.194+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ffbdc01c790 con 0x7ffbec06d630 2024-04-25T19:02:15.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.195+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbec076a40 con 0x7ffbec06d630 2024-04-25T19:02:15.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.198+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffbdc013470 con 0x7ffbec06d630 2024-04-25T19:02:15.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.277+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45334943-7f6e-4b40-8809-22b648c39b7d", "id": 0} v 0) -- 0x7ffbec0d4030 con 0x7ffbec06d630 2024-04-25T19:02:15.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.284+0000 7ffbe8c00640 1 -- 172.21.15.146:0/1638301884 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45334943-7f6e-4b40-8809-22b648c39b7d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ffbdc005990 con 0x7ffbec06d630 2024-04-25T19:02:15.287 INFO:teuthology.orchestra.run.smithi146.stdout:0 2024-04-25T19:02:15.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 msgr2=0x7ffbec1489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00e870 tx=0x7ffbdc00ed30 comp rx=0 tx=0).stop 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 shutdown_connections 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbec0b91f0 0x7ffbec159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbec0b4c10 0x7ffbec1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 --2- 172.21.15.146:0/1638301884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbec06d630 0x7ffbec1489b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 >> 172.21.15.146:0/1638301884 conn(0x7ffbec0b7690 msgr2=0x7ffbec0b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 shutdown_connections 2024-04-25T19:02:15.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.286+0000 7ffbf3200640 1 -- 172.21.15.146:0/1638301884 wait complete. 2024-04-25T19:02:15.312 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new d7e84f25-5ec3-4e15-9d9d-2208c882c785 1 2024-04-25T19:02:15.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.540+0000 7fc7ea600640 1 Processor -- start 2024-04-25T19:02:15.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.540+0000 7fc7ea600640 1 -- start start 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7ea600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7e40cfff0 0x7fc7e415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7ea600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e415e2e0 con 0x7fc7e406dc90 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7ea600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e40d05f0 con 0x7fc7e4143040 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7ea600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e40d1200 con 0x7fc7e40cfff0 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7e9200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7e40cfff0 0x7fc7e415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7e9200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7e40cfff0 0x7fc7e415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52948/0 (socket says 172.21.15.146:52948) 2024-04-25T19:02:15.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7e9200640 1 -- 172.21.15.146:0/2281005780 learned_addr learned my addr 172.21.15.146:0/2281005780 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.543+0000 7fc7dbe00640 1 --1- 172.21.15.146:0/2281005780 >> v1:172.21.15.146:6789/0 conn(0x7fc7e4143040 0x7fc7e406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59868/0 (socket says 172.21.15.146:59868) 2024-04-25T19:02:15.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.544+0000 7fc7e9200640 1 -- 172.21.15.146:0/2281005780 >> v1:172.21.15.146:6789/0 conn(0x7fc7e4143040 legacy=0x7fc7e406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.544+0000 7fc7e9200640 1 -- 172.21.15.146:0/2281005780 >> v1:172.21.15.123:6789/0 conn(0x7fc7e406dc90 legacy=0x7fc7e40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.544+0000 7fc7e9200640 1 -- 172.21.15.146:0/2281005780 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40d08d0 con 0x7fc7e40cfff0 2024-04-25T19:02:15.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.545+0000 7fc7e9200640 1 --2- 172.21.15.146:0/2281005780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e415dda0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953e16ea69af03ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.545+0000 7fc7e3e00640 1 -- 172.21.15.146:0/2281005780 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4037070 con 0x7fc7e40cfff0 2024-04-25T19:02:15.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.545+0000 7fc7e3e00640 1 -- 172.21.15.146:0/2281005780 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4032440 con 0x7fc7e40cfff0 2024-04-25T19:02:15.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.545+0000 7fc7e3e00640 1 -- 172.21.15.146:0/2281005780 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4032810 con 0x7fc7e40cfff0 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 -- 172.21.15.146:0/2281005780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 msgr2=0x7fc7e415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 --2- 172.21.15.146:0/2281005780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e415dda0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d402a270 comp rx=0 tx=0).stop 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 -- 172.21.15.146:0/2281005780 shutdown_connections 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 --2- 172.21.15.146:0/2281005780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e415dda0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 -- 172.21.15.146:0/2281005780 >> 172.21.15.146:0/2281005780 conn(0x7fc7e4073090 msgr2=0x7fc7e40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 -- 172.21.15.146:0/2281005780 shutdown_connections 2024-04-25T19:02:15.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.546+0000 7fc7ea600640 1 -- 172.21.15.146:0/2281005780 wait complete. 2024-04-25T19:02:15.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.548+0000 7fc7ea600640 1 Processor -- start 2024-04-25T19:02:15.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.548+0000 7fc7ea600640 1 -- start start 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7e4143040 0x7fc7e4153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 0x7fc7e406c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4143450 con 0x7fc7e40b0b30 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40d0e90 con 0x7fc7e40cfff0 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7ea600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40cb140 con 0x7fc7e4143040 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7e9200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7dbe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7e4143040 0x7fc7e4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7e9200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52958/0 (socket says 172.21.15.146:52958) 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.550+0000 7fc7e9200640 1 -- 172.21.15.146:0/1652660163 learned_addr learned my addr 172.21.15.146:0/1652660163 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9c00640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 0x7fc7e406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 -- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7e4143040 msgr2=0x7fc7e4153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7e4143040 0x7fc7e4153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 -- 172.21.15.146:0/1652660163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 msgr2=0x7fc7e406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 0x7fc7e406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 -- 172.21.15.146:0/1652660163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40ae440 con 0x7fc7e40cfff0 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9c00640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 0x7fc7e406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:15.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e9200640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4037040 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4042070 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e40ae720 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.551+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4030bb0 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.552+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e40aece0 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.552+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7d4047460 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.552+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc7d403db60 con 0x7fc7e40cfff0 2024-04-25T19:02:15.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.553+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4050740 con 0x7fc7e40cfff0 2024-04-25T19:02:15.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.555+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc7d403dd60 con 0x7fc7e40cfff0 2024-04-25T19:02:15.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.635+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d7e84f25-5ec3-4e15-9d9d-2208c882c785", "id": 1} v 0) -- 0x7fc7e40742b0 con 0x7fc7e40cfff0 2024-04-25T19:02:15.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.644+0000 7fc7e1600640 1 -- 172.21.15.146:0/1652660163 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7e84f25-5ec3-4e15-9d9d-2208c882c785", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc7d4030d50 con 0x7fc7e40cfff0 2024-04-25T19:02:15.646 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 msgr2=0x7fc7e40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 shutdown_connections 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e40b0b30 0x7fc7e406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7e4143040 0x7fc7e4153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 --2- 172.21.15.146:0/1652660163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e40cfff0 0x7fc7e40b4cf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.645+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 >> 172.21.15.146:0/1652660163 conn(0x7fc7e4073090 msgr2=0x7fc7e415f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.646+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 shutdown_connections 2024-04-25T19:02:15.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.646+0000 7fc7ea600640 1 -- 172.21.15.146:0/1652660163 wait complete. 2024-04-25T19:02:15.673 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 61e5a24c-4380-4098-8dce-72bf1dff941b 2 2024-04-25T19:02:15.719 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:15.717+0000 7f20e2a35040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:02:15.720 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:15.718+0000 7f359bc2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:02:15.821 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:15.819+0000 7f20e2a35040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:15.835 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:15.832+0000 7f359bc2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:15.870 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:15.869+0000 7f20e2a35040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:02:15.891 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:15.889+0000 7f359bc2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:02:15.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.917+0000 7f45e3800640 1 Processor -- start 2024-04-25T19:02:15.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.917+0000 7f45e3800640 1 -- start start 2024-04-25T19:02:15.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e3800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45dc0b6800 0x7f45dc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e3800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e3800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc164a50 con 0x7f45dc072ea0 2024-04-25T19:02:15.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e3800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc0d2e90 con 0x7f45dc0b6800 2024-04-25T19:02:15.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e3800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc0d0080 con 0x7f45dc0b8300 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.919+0000 7f45e1800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:46816/0 (socket says 172.21.15.146:46816) 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 -- 172.21.15.146:0/3968538841 learned_addr learned my addr 172.21.15.146:0/3968538841 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e0e00640 1 --2- 172.21.15.146:0/3968538841 >> v2:172.21.15.123:3300/0 conn(0x7f45dc0b6800 0x7f45dc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 -- 172.21.15.146:0/3968538841 >> v2:172.21.15.123:3300/0 conn(0x7f45dc0b6800 msgr2=0x7f45dc1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 --2- 172.21.15.146:0/3968538841 >> v2:172.21.15.123:3300/0 conn(0x7f45dc0b6800 0x7f45dc1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 -- 172.21.15.146:0/3968538841 >> v1:172.21.15.123:6789/0 conn(0x7f45dc072ea0 legacy=0x7f45dc0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:15.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.920+0000 7f45e1800640 1 -- 172.21.15.146:0/3968538841 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc164d30 con 0x7f45dc0b8300 2024-04-25T19:02:15.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.921+0000 7f45e1800640 1 --2- 172.21.15.146:0/3968538841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e350160cb072806a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.921+0000 7f45db400640 1 -- 172.21.15.146:0/3968538841 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4036070 con 0x7f45dc0b8300 2024-04-25T19:02:15.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.921+0000 7f45db400640 1 -- 172.21.15.146:0/3968538841 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d4031600 con 0x7f45dc0b8300 2024-04-25T19:02:15.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.921+0000 7f45db400640 1 -- 172.21.15.146:0/3968538841 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40319d0 con 0x7f45dc0b8300 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 -- 172.21.15.146:0/3968538841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 msgr2=0x7f45dc0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 --2- 172.21.15.146:0/3968538841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d402a630 comp rx=0 tx=0).stop 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 -- 172.21.15.146:0/3968538841 shutdown_connections 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 --2- 172.21.15.146:0/3968538841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b8300 0x7f45dc0b8710 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 --2- 172.21.15.146:0/3968538841 >> v2:172.21.15.123:3300/0 conn(0x7f45dc0b6800 0x7f45dc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 -- 172.21.15.146:0/3968538841 >> 172.21.15.146:0/3968538841 conn(0x7f45dc0d39b0 msgr2=0x7f45dc0d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 -- 172.21.15.146:0/3968538841 shutdown_connections 2024-04-25T19:02:15.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.922+0000 7f45e3800640 1 -- 172.21.15.146:0/3968538841 wait complete. 2024-04-25T19:02:15.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.925+0000 7f45e3800640 1 Processor -- start 2024-04-25T19:02:15.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.925+0000 7f45e3800640 1 -- start start 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 0x7f45dc1ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 0x7f45dc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc0b98e0 con 0x7f45dc072ea0 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc0cfd10 con 0x7f45dc0b8300 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e3800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc0d3230 con 0x7f45dc0b6800 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45dbe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 0x7f45dc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52966/0 (socket says 172.21.15.146:52966) 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 -- 172.21.15.146:0/2554723439 learned_addr learned my addr 172.21.15.146:0/2554723439 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:15.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e0e00640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 0x7f45dc1ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 -- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 msgr2=0x7f45dc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 0x7f45dc15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 -- 172.21.15.146:0/2554723439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 msgr2=0x7f45dc1ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 0x7f45dc1ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e1800640 1 -- 172.21.15.146:0/2554723439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc160e50 con 0x7f45dc0b8300 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.927+0000 7f45e0e00640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 0x7f45dc1ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45dbe00640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 0x7f45dc15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:15.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45e1800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4036050 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d4042070 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4009510 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc161130 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.928+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc162c90 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.929+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45d4007bd0 con 0x7f45dc0b8300 2024-04-25T19:02:15.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.929+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f45d4004530 con 0x7f45dc0b8300 2024-04-25T19:02:15.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.929+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc162ef0 con 0x7f45dc0b8300 2024-04-25T19:02:15.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:15.934+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f45d404b020 con 0x7f45dc0b8300 2024-04-25T19:02:16.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.011+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "61e5a24c-4380-4098-8dce-72bf1dff941b", "id": 2} v 0) -- 0x7f45dc0d3010 con 0x7f45dc0b8300 2024-04-25T19:02:16.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.019+0000 7f45d8c00640 1 -- 172.21.15.146:0/2554723439 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61e5a24c-4380-4098-8dce-72bf1dff941b", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f45d40047d0 con 0x7f45dc0b8300 2024-04-25T19:02:16.021 INFO:teuthology.orchestra.run.smithi146.stdout:2 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 msgr2=0x7f45dc15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009e50 tx=0x7f45d4004a40 comp rx=0 tx=0).stop 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 shutdown_connections 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45dc0b8300 0x7f45dc15db80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45dc0b6800 0x7f45dc15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 --2- 172.21.15.146:0/2554723439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45dc072ea0 0x7f45dc1ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 >> 172.21.15.146:0/2554723439 conn(0x7f45dc0d39b0 msgr2=0x7f45dc15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.020+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 shutdown_connections 2024-04-25T19:02:16.023 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.021+0000 7f45e3800640 1 -- 172.21.15.146:0/2554723439 wait complete. 2024-04-25T19:02:16.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 031b0f37-d4ee-4e43-8a3a-453a94c52f65 3 2024-04-25T19:02:16.063 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:16.061+0000 7f20e2a35040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:02:16.105 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:16.102+0000 7f359bc2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:02:16.117 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:16.116+0000 7f20e2a35040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:02:16.167 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:16.165+0000 7f359bc2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:02:16.177 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:16.176+0000 7f20e2a35040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:02:16.233 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:16.230+0000 7f359bc2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:02:16.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.298+0000 7f1691200640 1 Processor -- start 2024-04-25T19:02:16.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.298+0000 7f1691200640 1 -- start start 2024-04-25T19:02:16.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f1691200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f168c0d11a0 0x7f168c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f1691200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168c06cc00 con 0x7f168c06dca0 2024-04-25T19:02:16.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f1691200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168c06cee0 con 0x7f168c163d00 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f1691200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f168c06d1c0 con 0x7f168c0d11a0 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f168be00640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f168c163d00 0x7f168c0b90a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59876/0 (socket says 172.21.15.146:59876) 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f168be00640 1 -- 172.21.15.146:0/3593904318 learned_addr learned my addr 172.21.15.146:0/3593904318 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f168aa00640 1 --2- 172.21.15.146:0/3593904318 >> v2:172.21.15.123:3300/0 conn(0x7f168c0d11a0 0x7f168c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164026315 0 0) 0x7f168c06cee0 con 0x7f168c163d00 2024-04-25T19:02:16.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.300+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1670003520 con 0x7f168c163d00 2024-04-25T19:02:16.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2454840885 0 0) 0x7f1670003520 con 0x7f168c163d00 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 >> v2:172.21.15.123:3300/0 conn(0x7f168c0d11a0 msgr2=0x7f168c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 --2- 172.21.15.146:0/3593904318 >> v2:172.21.15.123:3300/0 conn(0x7f168c0d11a0 0x7f168c06c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 >> v1:172.21.15.123:6789/0 conn(0x7f168c06dca0 legacy=0x7f168c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c06d4a0 con 0x7f168c163d00 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f1684001670 con 0x7f168c163d00 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 >> v1:172.21.15.146:6789/0 conn(0x7f168c163d00 legacy=0x7f168c0b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f1670003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1670004800 0x7f1670004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16700051c0 0x7f167000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f168c06d1c0 con 0x7f16700051c0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f167000c160 con 0x7f16700039b0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f167000c360 con 0x7f1670004800 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168b400640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f1670003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168aa00640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1670004800 0x7f1670004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.301+0000 7f168be00640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16700051c0 0x7f167000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168b400640 1 -- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1670004800 msgr2=0x7f1670004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168b400640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1670004800 0x7f1670004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168b400640 1 -- 172.21.15.146:0/3593904318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16700051c0 msgr2=0x7f167000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168b400640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16700051c0 0x7f167000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168b400640 1 -- 172.21.15.146:0/3593904318 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1670003520 con 0x7f16700039b0 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168be00640 1 -- 172.21.15.146:0/3593904318 reap_dead start 2024-04-25T19:02:16.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.302+0000 7f168aa00640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1670004800 0x7f1670004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:16.307 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:16.303+0000 7f20e2a35040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.303+0000 7f168b400640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f1670003e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f167c001e50 tx=0x7f1670005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be688934887098d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.303+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c001a70 con 0x7f16700039b0 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.303+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f167c02ad80 con 0x7f16700039b0 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.303+0000 7f168a000640 1 -- 172.21.15.146:0/3593904318 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c003bb0 con 0x7f16700039b0 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 -- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 msgr2=0x7f1670003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f1670003e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f167c001e50 tx=0x7f1670005830 comp rx=0 tx=0).stop 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 -- 172.21.15.146:0/3593904318 shutdown_connections 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 --2- 172.21.15.146:0/3593904318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f1670003e20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 -- 172.21.15.146:0/3593904318 >> 172.21.15.146:0/3593904318 conn(0x7f168c0d6510 msgr2=0x7f168c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 -- 172.21.15.146:0/3593904318 shutdown_connections 2024-04-25T19:02:16.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.304+0000 7f1691200640 1 -- 172.21.15.146:0/3593904318 wait complete. 2024-04-25T19:02:16.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.306+0000 7f1691200640 1 Processor -- start 2024-04-25T19:02:16.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.306+0000 7f1691200640 1 -- start start 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168c154ec0 0x7f168c161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 0x7f168c1ea760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f168c06c570 con 0x7f168c154ec0 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f168c0b89f0 con 0x7f16700039b0 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f1691200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f168c06c6f0 con 0x7f168c1552d0 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f168b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f168b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:52996/0 (socket says 172.21.15.146:52996) 2024-04-25T19:02:16.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f168b400640 1 -- 172.21.15.146:0/3927431075 learned_addr learned my addr 172.21.15.146:0/3927431075 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f168be00640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 0x7f168c1ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.308+0000 7f168aa00640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168c154ec0 0x7f168c161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 -- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 msgr2=0x7f168c1ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 0x7f168c1ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 -- 172.21.15.146:0/3927431075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168c154ec0 msgr2=0x7f168c161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168c154ec0 0x7f168c161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 -- 172.21.15.146:0/3927431075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c156df0 con 0x7f16700039b0 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168be00640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 0x7f168c1ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:16.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f168b400640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f167c005040 tx=0x7f167c0014a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c037040 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f167c0050d0 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c007db0 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.309+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c1609a0 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.310+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c160f60 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.310+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f167c0054c0 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.310+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f167c007900 con 0x7f16700039b0 2024-04-25T19:02:16.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.311+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168c050740 con 0x7f16700039b0 2024-04-25T19:02:16.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.313+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f167c007040 con 0x7f16700039b0 2024-04-25T19:02:16.369 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:16.367+0000 7f359bc2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:16.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.392+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "031b0f37-d4ee-4e43-8a3a-453a94c52f65", "id": 3} v 0) -- 0x7f168c0d0660 con 0x7f16700039b0 2024-04-25T19:02:16.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.399+0000 7f1683400640 1 -- 172.21.15.146:0/3927431075 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "031b0f37-d4ee-4e43-8a3a-453a94c52f65", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f167c007300 con 0x7f16700039b0 2024-04-25T19:02:16.401 INFO:teuthology.orchestra.run.smithi146.stdout:3 2024-04-25T19:02:16.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 msgr2=0x7f168c1568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f167c005040 tx=0x7f167c0014a0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 shutdown_connections 2024-04-25T19:02:16.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168c1552d0 0x7f168c1ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.403 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168c154ec0 0x7f168c161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 --2- 172.21.15.146:0/3927431075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16700039b0 0x7f168c1568b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 >> 172.21.15.146:0/3927431075 conn(0x7f168c0d6510 msgr2=0x7f168c0d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:16.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 shutdown_connections 2024-04-25T19:02:16.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.401+0000 7f1691200640 1 -- 172.21.15.146:0/3927431075 wait complete. 2024-04-25T19:02:16.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 0d80edc9-5cee-4293-9dcb-e78b49c9ee0d 4 2024-04-25T19:02:16.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.673+0000 7fee40a00640 1 Processor -- start 2024-04-25T19:02:16.675 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.673+0000 7fee40a00640 1 -- start start 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.675+0000 7fee40a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.675+0000 7fee40a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee3c163d00 0x7fee3c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.675+0000 7fee40a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee3c06cc00 con 0x7fee3c0d0a90 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.675+0000 7fee40a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee3c0ba0b0 con 0x7fee3c163d00 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.675+0000 7fee40a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c06cee0 con 0x7fee3c06dca0 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:46862/0 (socket says 172.21.15.146:46862) 2024-04-25T19:02:16.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 -- 172.21.15.146:0/238050378 learned_addr learned my addr 172.21.15.146:0/238050378 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:16.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 -- 172.21.15.146:0/238050378 >> v2:172.21.15.123:3300/0 conn(0x7fee3c163d00 msgr2=0x7fee3c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:16.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee33e00640 1 --2- 172.21.15.146:0/238050378 >> v2:172.21.15.123:3300/0 conn(0x7fee3c163d00 0x7fee3c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 --2- 172.21.15.146:0/238050378 >> v2:172.21.15.123:3300/0 conn(0x7fee3c163d00 0x7fee3c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 -- 172.21.15.146:0/238050378 >> v1:172.21.15.123:6789/0 conn(0x7fee3c0d0a90 legacy=0x7fee3c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:16.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.676+0000 7fee3b400640 1 -- 172.21.15.146:0/238050378 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c06d1c0 con 0x7fee3c06dca0 2024-04-25T19:02:16.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.677+0000 7fee3b400640 1 --2- 172.21.15.146:0/238050378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009870 tx=0x7fee2c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd65145ab8b6507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:16.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.678+0000 7fee3aa00640 1 -- 172.21.15.146:0/238050378 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c037070 con 0x7fee3c06dca0 2024-04-25T19:02:16.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.678+0000 7fee3aa00640 1 -- 172.21.15.146:0/238050378 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c032440 con 0x7fee3c06dca0 2024-04-25T19:02:16.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.678+0000 7fee3aa00640 1 -- 172.21.15.146:0/238050378 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c032810 con 0x7fee3c06dca0 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.678+0000 7fee40a00640 1 -- 172.21.15.146:0/238050378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 msgr2=0x7fee3c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.678+0000 7fee40a00640 1 --2- 172.21.15.146:0/238050378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009870 tx=0x7fee2c02a210 comp rx=0 tx=0).stop 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 -- 172.21.15.146:0/238050378 shutdown_connections 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 --2- 172.21.15.146:0/238050378 >> v2:172.21.15.123:3300/0 conn(0x7fee3c163d00 0x7fee3c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 --2- 172.21.15.146:0/238050378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c06dca0 0x7fee3c06c570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 -- 172.21.15.146:0/238050378 >> 172.21.15.146:0/238050378 conn(0x7fee3c0d6510 msgr2=0x7fee3c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 -- 172.21.15.146:0/238050378 shutdown_connections 2024-04-25T19:02:16.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.679+0000 7fee40a00640 1 -- 172.21.15.146:0/238050378 wait complete. 2024-04-25T19:02:16.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.681+0000 7fee40a00640 1 Processor -- start 2024-04-25T19:02:16.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.681+0000 7fee40a00640 1 -- start start 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 0x7fee3c0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 0x7fee3c1f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c0b89f0 con 0x7fee3c163d00 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c06d780 con 0x7fee3c06dca0 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee40a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c0b90a0 con 0x7fee3c0d0a90 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee3b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee3b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:53010/0 (socket says 172.21.15.146:53010) 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee33e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 0x7fee3c0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.683+0000 7fee3b400640 1 -- 172.21.15.146:0/1544503294 learned_addr learned my addr 172.21.15.146:0/1544503294 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3be00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 0x7fee3c1f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 -- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 msgr2=0x7fee3c0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 0x7fee3c0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 -- 172.21.15.146:0/1544503294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 msgr2=0x7fee3c1f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 0x7fee3c1f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 -- 172.21.15.146:0/1544503294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c0c9ed0 con 0x7fee3c06dca0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee33e00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 0x7fee3c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3be00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 0x7fee3c1f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee3b400640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009870 tx=0x7fee2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.684+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c041020 con 0x7fee3c06dca0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c030970 con 0x7fee3c06dca0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c037040 con 0x7fee3c06dca0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c0ca1b0 con 0x7fee3c06dca0 2024-04-25T19:02:16.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c0ca770 con 0x7fee3c06dca0 2024-04-25T19:02:16.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee2c044020 con 0x7fee3c06dca0 2024-04-25T19:02:16.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.685+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fee2c030e00 con 0x7fee3c06dca0 2024-04-25T19:02:16.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.686+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c050740 con 0x7fee3c06dca0 2024-04-25T19:02:16.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.688+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fee2c04b030 con 0x7fee3c06dca0 2024-04-25T19:02:16.754 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:16.752+0000 7f20e2a35040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:02:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.768+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0d80edc9-5cee-4293-9dcb-e78b49c9ee0d", "id": 4} v 0) -- 0x7fee3c0d0660 con 0x7fee3c06dca0 2024-04-25T19:02:16.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.779+0000 7fee33400640 1 -- 172.21.15.146:0/1544503294 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d80edc9-5cee-4293-9dcb-e78b49c9ee0d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fee2c046400 con 0x7fee3c06dca0 2024-04-25T19:02:16.781 INFO:teuthology.orchestra.run.smithi146.stdout:4 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 msgr2=0x7fee3c0ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009870 tx=0x7fee2c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 shutdown_connections 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee3c163d00 0x7fee3c1f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3c0d0a90 0x7fee3c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 --2- 172.21.15.146:0/1544503294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee3c06dca0 0x7fee3c0ceba0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 >> 172.21.15.146:0/1544503294 conn(0x7fee3c0d6510 msgr2=0x7fee3c15e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.780+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 shutdown_connections 2024-04-25T19:02:16.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:16.781+0000 7fee40a00640 1 -- 172.21.15.146:0/1544503294 wait complete. 2024-04-25T19:02:16.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 18466f32-ab3c-49ce-80ce-e298e4e0a0de 5 2024-04-25T19:02:16.856 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:16.854+0000 7f359bc2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:02:17.046 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.044+0000 7f20e2a35040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:02:17.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.056+0000 7f4042000640 1 Processor -- start 2024-04-25T19:02:17.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.056+0000 7f4042000640 1 -- start start 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.058+0000 7f4042000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 0x7f403c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.058+0000 7f4042000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f403c163d00 0x7f403c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.058+0000 7f4042000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0d0a90 0x7f403c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4042000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f403c0d13e0 con 0x7f403c06dca0 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4042000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f403c06cc00 con 0x7f403c163d00 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4042000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f403c06cee0 con 0x7f403c0d0a90 2024-04-25T19:02:17.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f403c163d00 0x7f403c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f403c163d00 0x7f403c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45004/0 (socket says 172.21.15.146:45004) 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 -- 172.21.15.146:0/886943977 learned_addr learned my addr 172.21.15.146:0/886943977 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4041600640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0d0a90 0x7f403c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4040c00640 1 --2- 172.21.15.146:0/886943977 >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 0x7f403c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 -- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0d0a90 msgr2=0x7f403c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0d0a90 0x7f403c0d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 -- 172.21.15.146:0/886943977 >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 msgr2=0x7f403c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 --2- 172.21.15.146:0/886943977 >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 0x7f403c06c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f403be00640 1 -- 172.21.15.146:0/886943977 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c06d1c0 con 0x7f403c163d00 2024-04-25T19:02:17.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.059+0000 7f4040c00640 1 --2- 172.21.15.146:0/886943977 >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 0x7f403c06c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:17.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.060+0000 7f403be00640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c163d00 0x7f403c0b4c10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d0030 tx=0x7f402802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef70036588461db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.061+0000 7f403b400640 1 -- 172.21.15.146:0/886943977 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028037070 con 0x7f403c163d00 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.061+0000 7f403b400640 1 -- 172.21.15.146:0/886943977 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4028032440 con 0x7f403c163d00 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.061+0000 7f403b400640 1 -- 172.21.15.146:0/886943977 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028032810 con 0x7f403c163d00 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.061+0000 7f4042000640 1 -- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c163d00 msgr2=0x7f403c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.061+0000 7f4042000640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c163d00 0x7f403c0b4c10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d0030 tx=0x7f402802a1e0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 -- 172.21.15.146:0/886943977 shutdown_connections 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0d0a90 0x7f403c0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 --2- 172.21.15.146:0/886943977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c163d00 0x7f403c0b4c10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 --2- 172.21.15.146:0/886943977 >> v2:172.21.15.123:3300/0 conn(0x7f403c06dca0 0x7f403c06c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 -- 172.21.15.146:0/886943977 >> 172.21.15.146:0/886943977 conn(0x7f403c0d6510 msgr2=0x7f403c0d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 -- 172.21.15.146:0/886943977 shutdown_connections 2024-04-25T19:02:17.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.062+0000 7f4042000640 1 -- 172.21.15.146:0/886943977 wait complete. 2024-04-25T19:02:17.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.064+0000 7f4042000640 1 Processor -- start 2024-04-25T19:02:17.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.064+0000 7f4042000640 1 -- start start 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c06dca0 0x7f403c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 0x7f403c0baa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0b89f0 con 0x7f403c163d00 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f403c06d780 con 0x7f403c0d0a90 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4042000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0d46b0 con 0x7f403c06dca0 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4040c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c06dca0 0x7f403c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45016/0 (socket says 172.21.15.146:45016) 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 -- 172.21.15.146:0/1523814120 learned_addr learned my addr 172.21.15.146:0/1523814120 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4041600640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 0x7f403c0baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 -- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c06dca0 msgr2=0x7f403c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c06dca0 0x7f403c14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 -- 172.21.15.146:0/1523814120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 msgr2=0x7f403c0baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 0x7f403c0baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f403be00640 1 -- 172.21.15.146:0/1523814120 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0cf120 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.066+0000 7f4041600640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 0x7f403c0baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f403be00640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f402802a220 tx=0x7f40280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028037040 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4028042070 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0cf400 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028030880 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c0cf9c0 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.067+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4028030a20 con 0x7f403c0d0a90 2024-04-25T19:02:17.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.068+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4028030d10 con 0x7f403c0d0a90 2024-04-25T19:02:17.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.068+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c050740 con 0x7f403c0d0a90 2024-04-25T19:02:17.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.073+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4028047460 con 0x7f403c0d0a90 2024-04-25T19:02:17.106 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.104+0000 7f20e2a35040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:02:17.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.152+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "18466f32-ab3c-49ce-80ce-e298e4e0a0de", "id": 5} v 0) -- 0x7f403c0d0660 con 0x7f403c0d0a90 2024-04-25T19:02:17.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.162+0000 7f4038c00640 1 -- 172.21.15.146:0/1523814120 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "18466f32-ab3c-49ce-80ce-e298e4e0a0de", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4028047640 con 0x7f403c0d0a90 2024-04-25T19:02:17.164 INFO:teuthology.orchestra.run.smithi146.stdout:5 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.163+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 msgr2=0x7f403c14d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.163+0000 7f4042000640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f402802a220 tx=0x7f40280046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 shutdown_connections 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f403c163d00 0x7f403c0baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c0d0a90 0x7f403c14d370 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 --2- 172.21.15.146:0/1523814120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c06dca0 0x7f403c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 >> 172.21.15.146:0/1523814120 conn(0x7f403c0d6510 msgr2=0x7f403c0b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 shutdown_connections 2024-04-25T19:02:17.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.164+0000 7f4042000640 1 -- 172.21.15.146:0/1523814120 wait complete. 2024-04-25T19:02:17.173 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.171+0000 7f359bc2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:02:17.188 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 87be9fd7-46a7-45f4-afa8-d04439198a81 6 2024-04-25T19:02:17.237 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.235+0000 7f20e2a35040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:02:17.238 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.236+0000 7f359bc2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:02:17.378 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.376+0000 7f359bc2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:02:17.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.440+0000 7f63aae00640 1 Processor -- start 2024-04-25T19:02:17.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.440+0000 7f63aae00640 1 -- start start 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.441+0000 7f63aae00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.441+0000 7f63aae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f63a4163d00 0x7f63a40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63aae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a406cc00 con 0x7f63a40d0a90 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63aae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f63a40ba0b0 con 0x7f63a4163d00 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63aae00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63a406cee0 con 0x7f63a406dca0 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:36932/0 (socket says 172.21.15.146:36932) 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 -- 172.21.15.146:0/3320803861 learned_addr learned my addr 172.21.15.146:0/3320803861 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9000640 1 --2- 172.21.15.146:0/3320803861 >> v2:172.21.15.123:3300/0 conn(0x7f63a4163d00 0x7f63a40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 -- 172.21.15.146:0/3320803861 >> v2:172.21.15.123:3300/0 conn(0x7f63a4163d00 msgr2=0x7f63a40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 --2- 172.21.15.146:0/3320803861 >> v2:172.21.15.123:3300/0 conn(0x7f63a4163d00 0x7f63a40b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 -- 172.21.15.146:0/3320803861 >> v1:172.21.15.123:6789/0 conn(0x7f63a40d0a90 legacy=0x7f63a40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.442+0000 7f63a9a00640 1 -- 172.21.15.146:0/3320803861 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a406d1c0 con 0x7f63a406dca0 2024-04-25T19:02:17.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.444+0000 7f63a9a00640 1 --2- 172.21.15.146:0/3320803861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6398009870 tx=0x7f639802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42947e053b9c073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.444+0000 7f63a3e00640 1 -- 172.21.15.146:0/3320803861 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398037070 con 0x7f63a406dca0 2024-04-25T19:02:17.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.444+0000 7f63a3e00640 1 -- 172.21.15.146:0/3320803861 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6398032440 con 0x7f63a406dca0 2024-04-25T19:02:17.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.444+0000 7f63a3e00640 1 -- 172.21.15.146:0/3320803861 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398032810 con 0x7f63a406dca0 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 -- 172.21.15.146:0/3320803861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 msgr2=0x7f63a406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 --2- 172.21.15.146:0/3320803861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6398009870 tx=0x7f639802a210 comp rx=0 tx=0).stop 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 -- 172.21.15.146:0/3320803861 shutdown_connections 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 --2- 172.21.15.146:0/3320803861 >> v2:172.21.15.123:3300/0 conn(0x7f63a4163d00 0x7f63a40b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 --2- 172.21.15.146:0/3320803861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a406dca0 0x7f63a406c570 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 -- 172.21.15.146:0/3320803861 >> 172.21.15.146:0/3320803861 conn(0x7f63a40d6510 msgr2=0x7f63a40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.445+0000 7f63aae00640 1 -- 172.21.15.146:0/3320803861 shutdown_connections 2024-04-25T19:02:17.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.446+0000 7f63aae00640 1 -- 172.21.15.146:0/3320803861 wait complete. 2024-04-25T19:02:17.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.448+0000 7f63aae00640 1 Processor -- start 2024-04-25T19:02:17.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.448+0000 7f63aae00640 1 -- start start 2024-04-25T19:02:17.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 0x7f63a4151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 0x7f63a40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63a4163d00 0x7f63a40c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63a40b89f0 con 0x7f63a4163d00 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63a406d780 con 0x7f63a406dca0 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.449+0000 7f63aae00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63a40b90a0 con 0x7f63a40d0a90 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 0x7f63a40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 0x7f63a4151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 0x7f63a40c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:36942/0 (socket says 172.21.15.146:36942) 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9000640 1 -- 172.21.15.146:0/4181366973 learned_addr learned my addr 172.21.15.146:0/4181366973 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63aa400640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63a4163d00 0x7f63a40c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 -- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 msgr2=0x7f63a40c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 0x7f63a40c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 -- 172.21.15.146:0/4181366973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63a4163d00 msgr2=0x7f63a40c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63a4163d00 0x7f63a40c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 -- 172.21.15.146:0/4181366973 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a4150280 con 0x7f63a406dca0 2024-04-25T19:02:17.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.450+0000 7f63a9a00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 0x7f63a4151320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63a406d060 tx=0x7f63980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398041020 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6398030780 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398037040 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a4150560 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a4150b20 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6398044020 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.451+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f639803dc90 con 0x7f63a406dca0 2024-04-25T19:02:17.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.452+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a4050740 con 0x7f63a406dca0 2024-04-25T19:02:17.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.454+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f639804b030 con 0x7f63a406dca0 2024-04-25T19:02:17.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.533+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "87be9fd7-46a7-45f4-afa8-d04439198a81", "id": 6} v 0) -- 0x7f63a40d0660 con 0x7f63a406dca0 2024-04-25T19:02:17.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.544+0000 7f63a1600640 1 -- 172.21.15.146:0/4181366973 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87be9fd7-46a7-45f4-afa8-d04439198a81", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f639803d420 con 0x7f63a406dca0 2024-04-25T19:02:17.547 INFO:teuthology.orchestra.run.smithi146.stdout:6 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 msgr2=0x7f63a4151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 0x7f63a4151320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63a406d060 tx=0x7f63980046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 shutdown_connections 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63a4163d00 0x7f63a40c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63a40d0a90 0x7f63a40c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 --2- 172.21.15.146:0/4181366973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a406dca0 0x7f63a4151320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 >> 172.21.15.146:0/4181366973 conn(0x7f63a40d6510 msgr2=0x7f63a40ba2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 shutdown_connections 2024-04-25T19:02:17.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.545+0000 7f63aae00640 1 -- 172.21.15.146:0/4181366973 wait complete. 2024-04-25T19:02:17.565 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.563+0000 7f20e2a35040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:02:17.569 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new ad614261-aee1-4cc1-83b5-874100e98720 7 2024-04-25T19:02:17.620 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.618+0000 7f20e2a35040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:02:17.732 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.730+0000 7f359bc2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:02:17.792 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.789+0000 7f359bc2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:02:17.797 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.795+0000 7f20e2a35040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:02:17.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.810+0000 7f8c5ee00640 1 Processor -- start 2024-04-25T19:02:17.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.811+0000 7f8c5ee00640 1 -- start start 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5ee00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c5806dca0 0x7f8c5806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5ee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 0x7f8c580b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c5806cc00 con 0x7f8c58163d00 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5ee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8c580ba0b0 con 0x7f8c580d11a0 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5ee00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5806cee0 con 0x7f8c5806dca0 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5da00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c5806dca0 0x7f8c5806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5da00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c5806dca0 0x7f8c5806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45038/0 (socket says 172.21.15.146:45038) 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5da00640 1 -- 172.21.15.146:0/3241368162 learned_addr learned my addr 172.21.15.146:0/3241368162 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.813+0000 7f8c5e400640 1 --2- 172.21.15.146:0/3241368162 >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 0x7f8c580b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.814+0000 7f8c5da00640 1 -- 172.21.15.146:0/3241368162 >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 msgr2=0x7f8c580b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.814+0000 7f8c5da00640 1 --2- 172.21.15.146:0/3241368162 >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 0x7f8c580b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.814+0000 7f8c5da00640 1 -- 172.21.15.146:0/3241368162 >> v1:172.21.15.123:6789/0 conn(0x7f8c58163d00 legacy=0x7f8c580d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.814+0000 7f8c5da00640 1 -- 172.21.15.146:0/3241368162 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5806d1c0 con 0x7f8c5806dca0 2024-04-25T19:02:17.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.814+0000 7f8c5e400640 1 --2- 172.21.15.146:0/3241368162 >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 0x7f8c580b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:17.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.815+0000 7f8c5da00640 1 --2- 172.21.15.146:0/3241368162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c5806c570 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c5808e6b0 tx=0x7f8c4c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55fe619a077d709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.815+0000 7f8c57e00640 1 -- 172.21.15.146:0/3241368162 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c037070 con 0x7f8c5806dca0 2024-04-25T19:02:17.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.815+0000 7f8c57e00640 1 -- 172.21.15.146:0/3241368162 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c032440 con 0x7f8c5806dca0 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.815+0000 7f8c57e00640 1 -- 172.21.15.146:0/3241368162 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c032810 con 0x7f8c5806dca0 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 -- 172.21.15.146:0/3241368162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 msgr2=0x7f8c5806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/3241368162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c5806c570 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c5808e6b0 tx=0x7f8c4c02a270 comp rx=0 tx=0).stop 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 -- 172.21.15.146:0/3241368162 shutdown_connections 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/3241368162 >> v2:172.21.15.123:3300/0 conn(0x7f8c580d11a0 0x7f8c580b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/3241368162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c5806c570 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 -- 172.21.15.146:0/3241368162 >> 172.21.15.146:0/3241368162 conn(0x7f8c580d6510 msgr2=0x7f8c580d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 -- 172.21.15.146:0/3241368162 shutdown_connections 2024-04-25T19:02:17.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.816+0000 7f8c5ee00640 1 -- 172.21.15.146:0/3241368162 wait complete. 2024-04-25T19:02:17.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.818+0000 7f8c5ee00640 1 Processor -- start 2024-04-25T19:02:17.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.819+0000 7f8c5ee00640 1 -- start start 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c58163d00 0x7f8c58161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c58155b00 0x7f8c58161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c580b89f0 con 0x7f8c58163d00 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5806d780 con 0x7f8c5806dca0 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5ee00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c580b90a0 con 0x7f8c58155b00 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5da00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5da00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:45054/0 (socket says 172.21.15.146:45054) 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5da00640 1 -- 172.21.15.146:0/667969429 learned_addr learned my addr 172.21.15.146:0/667969429 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T19:02:17.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.820+0000 7f8c5e400640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c58155b00 0x7f8c58161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 -- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c58155b00 msgr2=0x7f8c58161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c58155b00 0x7f8c58161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 -- 172.21.15.146:0/667969429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c58163d00 msgr2=0x7f8c58161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c58163d00 0x7f8c58161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 -- 172.21.15.146:0/667969429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c581eb510 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.821+0000 7f8c5da00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02a2b0 tx=0x7f8c4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c037040 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c042070 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c581eb7f0 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c030bb0 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581ebdb0 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c4c047460 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.822+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8c4c03daa0 con 0x7f8c5806dca0 2024-04-25T19:02:17.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.823+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c58050740 con 0x7f8c5806dca0 2024-04-25T19:02:17.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.825+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8c4c03dca0 con 0x7f8c5806dca0 2024-04-25T19:02:17.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.905+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad614261-aee1-4cc1-83b5-874100e98720", "id": 7} v 0) -- 0x7f8c580d0660 con 0x7f8c5806dca0 2024-04-25T19:02:17.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.917+0000 7f8c55600640 1 -- 172.21.15.146:0/667969429 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad614261-aee1-4cc1-83b5-874100e98720", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8c4c03d440 con 0x7f8c5806dca0 2024-04-25T19:02:17.919 INFO:teuthology.orchestra.run.smithi146.stdout:7 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 msgr2=0x7f8c58160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02a2b0 tx=0x7f8c4c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 shutdown_connections 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c58155b00 0x7f8c58161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c58163d00 0x7f8c58161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 --2- 172.21.15.146:0/667969429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5806dca0 0x7f8c58160ad0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 >> 172.21.15.146:0/667969429 conn(0x7f8c580d6510 msgr2=0x7f8c580bb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 shutdown_connections 2024-04-25T19:02:17.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:02:17.918+0000 7f8c5ee00640 1 -- 172.21.15.146:0/667969429 wait complete. 2024-04-25T19:02:17.944 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T19:02:17.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T19:02:17.951 INFO:tasks.ceph.osd.0:Started 2024-04-25T19:02:17.951 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T19:02:17.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T19:02:17.959 INFO:tasks.ceph.osd.1:Started 2024-04-25T19:02:17.959 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T19:02:17.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T19:02:17.965 INFO:tasks.ceph.osd.2:Started 2024-04-25T19:02:17.966 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T19:02:17.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T19:02:17.967 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:17.965+0000 7f20e2a35040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:02:17.971 INFO:tasks.ceph.osd.3:Started 2024-04-25T19:02:17.971 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T19:02:17.971 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T19:02:17.979 INFO:tasks.ceph.osd.4:Started 2024-04-25T19:02:17.979 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T19:02:17.979 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T19:02:17.986 INFO:tasks.ceph.osd.5:Started 2024-04-25T19:02:17.986 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T19:02:17.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T19:02:17.986 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:17.984+0000 7f359bc2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:02:17.990 INFO:tasks.ceph.osd.6:Started 2024-04-25T19:02:17.990 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T19:02:17.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T19:02:17.995 INFO:tasks.ceph.osd.7:Started 2024-04-25T19:02:17.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:18.044 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:18.042+0000 7f20e2a35040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:02:18.154 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:18.153+0000 7f20e2a35040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:18.215 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:18.213+0000 7f359bc2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:02:18.244 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:18.242+0000 7f20e2a35040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:02:18.305 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:18.303+0000 7f359bc2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:02:18.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.329+0000 7f2c06a00640 1 Processor -- start 2024-04-25T19:02:18.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.330+0000 7f2c06a00640 1 -- start start 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 0x7f2c00070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c00161b10 0x7f2c00161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 0x7f2c00154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2c000d1530 con 0x7f2c00161b10 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2c00162460 con 0x7f2c000d1120 2024-04-25T19:02:18.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.331+0000 7f2c06a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c000aec30 con 0x7f2c00154950 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c00161b10 0x7f2c00161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c00161b10 0x7f2c00161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33374/0 (socket says 172.21.15.123:33374) 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 -- 172.21.15.123:0/3590174192 learned_addr learned my addr 172.21.15.123:0/3590174192 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2c04a00640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 0x7f2c00154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bffe00640 1 --2- 172.21.15.123:0/3590174192 >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 0x7f2c00070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 -- 172.21.15.123:0/3590174192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 msgr2=0x7f2c00154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 0x7f2c00154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 -- 172.21.15.123:0/3590174192 >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 msgr2=0x7f2c00070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 --2- 172.21.15.123:0/3590174192 >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 0x7f2c00070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bff400640 1 -- 172.21.15.123:0/3590174192 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c000aef10 con 0x7f2c00161b10 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2c04a00640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 0x7f2c00154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:18.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.332+0000 7f2bffe00640 1 --2- 172.21.15.123:0/3590174192 >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 0x7f2c00070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.334+0000 7f2bff400640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c00161b10 0x7f2c00161f20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c000b6a60 tx=0x7f2bec02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6b2ad98beb7717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.334+0000 7f2bfea00640 1 -- 172.21.15.123:0/3590174192 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec037070 con 0x7f2c00161b10 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.334+0000 7f2bfea00640 1 -- 172.21.15.123:0/3590174192 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec02ae00 con 0x7f2c00161b10 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.334+0000 7f2bfea00640 1 -- 172.21.15.123:0/3590174192 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec032660 con 0x7f2c00161b10 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 -- 172.21.15.123:0/3590174192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c00161b10 msgr2=0x7f2c00161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:18.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c00161b10 0x7f2c00161f20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c000b6a60 tx=0x7f2bec02a0b0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 -- 172.21.15.123:0/3590174192 shutdown_connections 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00154950 0x7f2c00154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 --2- 172.21.15.123:0/3590174192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c00161b10 0x7f2c00161f20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 --2- 172.21.15.123:0/3590174192 >> v2:172.21.15.146:3300/0 conn(0x7f2c000d1120 0x7f2c00070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 -- 172.21.15.123:0/3590174192 >> 172.21.15.123:0/3590174192 conn(0x7f2c000d1ae0 msgr2=0x7f2c000d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 -- 172.21.15.123:0/3590174192 shutdown_connections 2024-04-25T19:02:18.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.335+0000 7f2c06a00640 1 -- 172.21.15.123:0/3590174192 wait complete. 2024-04-25T19:02:18.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.337+0000 7f2c06a00640 1 Processor -- start 2024-04-25T19:02:18.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.337+0000 7f2c06a00640 1 -- start start 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c00154950 0x7f2c000b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 0x7f2c00153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001552a0 con 0x7f2c000d1120 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000af4d0 con 0x7f2c00154950 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c06a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c000cc840 con 0x7f2c00161b10 2024-04-25T19:02:18.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33386/0 (socket says 172.21.15.123:33386) 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 -- 172.21.15.123:0/2276400946 learned_addr learned my addr 172.21.15.123:0/2276400946 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2c04a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 0x7f2c00153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bff400640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c00154950 0x7f2c000b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 -- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 msgr2=0x7f2c00153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 0x7f2c00153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 -- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c00154950 msgr2=0x7f2c000b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.339+0000 7f2bffe00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c00154950 0x7f2c000b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bffe00640 1 -- 172.21.15.123:0/2276400946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c000b6710 con 0x7f2c000d1120 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2c04a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 0x7f2c00153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bffe00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d800 tx=0x7f2bf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0018070 con 0x7f2c000d1120 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf0004030 con 0x7f2c000d1120 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0004400 con 0x7f2c000d1120 2024-04-25T19:02:18.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c000b47c0 con 0x7f2c000d1120 2024-04-25T19:02:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c000b4d80 con 0x7f2c000d1120 2024-04-25T19:02:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.340+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2bf0009a30 con 0x7f2c000d1120 2024-04-25T19:02:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.341+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2bf0009d20 con 0x7f2c000d1120 2024-04-25T19:02:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.341+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c000b7910 con 0x7f2c000d1120 2024-04-25T19:02:18.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.344+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2c000b7910 con 0x7f2c000d1120 2024-04-25T19:02:18.390 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:18.388+0000 7f359bc2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:18.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.406+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c00076a40 con 0x7f2c000d1120 2024-04-25T19:02:18.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.408+0000 7f2bf7e00640 1 -- 172.21.15.123:0/2276400946 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2bf0003610 con 0x7f2c000d1120 2024-04-25T19:02:18.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:18.410 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:17.908495+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 msgr2=0x7f2c001f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d800 tx=0x7f2bf000dcc0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 shutdown_connections 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c00161b10 0x7f2c00153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c00154950 0x7f2c000b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 --2- 172.21.15.123:0/2276400946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c000d1120 0x7f2c001f7890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 >> 172.21.15.123:0/2276400946 conn(0x7f2c000d1ae0 msgr2=0x7f2c000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 shutdown_connections 2024-04-25T19:02:18.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:18.409+0000 7f2c06a00640 1 -- 172.21.15.123:0/2276400946 wait complete. 2024-04-25T19:02:18.433 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T19:02:18.433 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T19:02:18.443 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:18.440+0000 7f359bc2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:02:18.731 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T19:02:18.729+0000 7fde36419640 -1 Falling back to public interface 2024-04-25T19:02:18.732 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T19:02:18.731+0000 7f0194a86640 -1 Falling back to public interface 2024-04-25T19:02:18.745 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T19:02:18.743+0000 7ffab3f55640 -1 Falling back to public interface 2024-04-25T19:02:18.745 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T19:02:18.743+0000 7fe2f1a0c640 -1 Falling back to public interface 2024-04-25T19:02:18.766 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T19:02:18.764+0000 7fbbf0ffc640 -1 Falling back to public interface 2024-04-25T19:02:18.781 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T19:02:18.779+0000 7f5ddc9fc640 -1 Falling back to public interface 2024-04-25T19:02:18.799 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:18.798+0000 7f20e2a35040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:02:18.806 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T19:02:18.804+0000 7f46ee40f640 -1 Falling back to public interface 2024-04-25T19:02:18.813 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T19:02:18.812+0000 7ff637f57640 -1 Falling back to public interface 2024-04-25T19:02:18.932 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:18.930+0000 7f20e2a35040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:02:18.997 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:18.995+0000 7f359bc2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:02:19.133 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T19:02:19.131+0000 7f359bc2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:02:19.238 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:19.478 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.476+0000 7f0345600640 1 Processor -- start 2024-04-25T19:02:19.478 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.476+0000 7f0345600640 1 -- start start 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0340164920 0x7f03400d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03400ae7e0 0x7f034006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f034006da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03400d1310 con 0x7f03400ae7e0 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f034006c710 con 0x7f0340164920 2024-04-25T19:02:19.480 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.478+0000 7f0345600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f034006c9f0 con 0x7f03400aebf0 2024-04-25T19:02:19.481 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03400ae7e0 0x7f034006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.481 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03400ae7e0 0x7f034006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33458/0 (socket says 172.21.15.123:33458) 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 -- 172.21.15.123:0/4113742537 learned_addr learned my addr 172.21.15.123:0/4113742537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 -- 172.21.15.123:0/4113742537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 msgr2=0x7f034006da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f033f400640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f034006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f033ea00640 1 --2- 172.21.15.123:0/4113742537 >> v2:172.21.15.146:3300/0 conn(0x7f0340164920 0x7f03400d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f034006da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 -- 172.21.15.123:0/4113742537 >> v2:172.21.15.146:3300/0 conn(0x7f0340164920 msgr2=0x7f03400d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 --2- 172.21.15.123:0/4113742537 >> v2:172.21.15.146:3300/0 conn(0x7f0340164920 0x7f03400d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.482 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.479+0000 7f0337e00640 1 -- 172.21.15.123:0/4113742537 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034006ccd0 con 0x7f03400ae7e0 2024-04-25T19:02:19.483 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.480+0000 7f0337e00640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034006d4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0340071250 tx=0x7f032802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eae50101165b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:19.483 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.481+0000 7f033e000640 1 -- 172.21.15.123:0/4113742537 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328037070 con 0x7f03400ae7e0 2024-04-25T19:02:19.483 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.481+0000 7f033e000640 1 -- 172.21.15.123:0/4113742537 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328032440 con 0x7f03400ae7e0 2024-04-25T19:02:19.483 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.481+0000 7f033e000640 1 -- 172.21.15.123:0/4113742537 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328032810 con 0x7f03400ae7e0 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 -- 172.21.15.123:0/4113742537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 msgr2=0x7f034006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034006d4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0340071250 tx=0x7f032802a1e0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 -- 172.21.15.123:0/4113742537 shutdown_connections 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f034006da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 --2- 172.21.15.123:0/4113742537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034006d4d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 --2- 172.21.15.123:0/4113742537 >> v2:172.21.15.146:3300/0 conn(0x7f0340164920 0x7f03400d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 -- 172.21.15.123:0/4113742537 >> 172.21.15.123:0/4113742537 conn(0x7f03400ce190 msgr2=0x7f03400ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:19.484 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 -- 172.21.15.123:0/4113742537 shutdown_connections 2024-04-25T19:02:19.485 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.482+0000 7f0345600640 1 -- 172.21.15.123:0/4113742537 wait complete. 2024-04-25T19:02:19.486 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.484+0000 7f0345600640 1 Processor -- start 2024-04-25T19:02:19.486 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.484+0000 7f0345600640 1 -- start start 2024-04-25T19:02:19.488 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.488 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f03401559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0340164920 0x7f0340156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0340168490 con 0x7f03400ae7e0 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0340168310 con 0x7f0340164920 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f0345600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0340168610 con 0x7f03400aebf0 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f033ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f033ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33462/0 (socket says 172.21.15.123:33462) 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.486+0000 7f033ea00640 1 -- 172.21.15.123:0/3834632390 learned_addr learned my addr 172.21.15.123:0/3834632390 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033f400640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0340164920 0x7f0340156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0337e00640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f03401559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 -- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 msgr2=0x7f03401559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f03401559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 -- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0340164920 msgr2=0x7f0340156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0340164920 0x7f0340156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.489 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 -- 172.21.15.123:0/3834632390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0340155f10 con 0x7f03400ae7e0 2024-04-25T19:02:19.490 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f033ea00640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f033000c5c0 tx=0x7f033000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330004230 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03300043d0 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034009d9f0 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330003850 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.487+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034009dfb0 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.488+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0340164510 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.489+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 2) ==== 48416+0+0 (secure 0 0 0) 0x7f03300039f0 con 0x7f03400ae7e0 2024-04-25T19:02:19.491 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.489+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f033004f500 con 0x7f03400ae7e0 2024-04-25T19:02:19.493 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.492+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0340164510 con 0x7f03400ae7e0 2024-04-25T19:02:19.559 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.558+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0340076a40 con 0x7f03400ae7e0 2024-04-25T19:02:19.561 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.559+0000 7f0336a00640 1 -- 172.21.15.123:0/3834632390 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0330015020 con 0x7f03400ae7e0 2024-04-25T19:02:19.561 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-25T19:02:19.561 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":10,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:17.908495+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 msgr2=0x7f034009edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f033000c5c0 tx=0x7f033000ca80 comp rx=0 tx=0).stop 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 shutdown_connections 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0340164920 0x7f0340156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03400aebf0 0x7f03401559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 --2- 172.21.15.123:0/3834632390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400ae7e0 0x7f034009edf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 >> 172.21.15.123:0/3834632390 conn(0x7f03400ce190 msgr2=0x7f0340163c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:19.562 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 shutdown_connections 2024-04-25T19:02:19.563 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:19.560+0000 7f0345600640 1 -- 172.21.15.123:0/3834632390 wait complete. 2024-04-25T19:02:19.581 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T19:02:20.001 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:19.999+0000 7f2081e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T19:02:20.002 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-25T19:02:20.000+0000 7f2081e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T19:02:21.978 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T19:02:21.976+0000 7f46ee40f640 -1 osd.6 0 log_to_monitors true 2024-04-25T19:02:22.105 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T19:02:22.104+0000 7fde36419640 -1 osd.2 0 log_to_monitors true 2024-04-25T19:02:22.395 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T19:02:22.393+0000 7f5ddc9fc640 -1 osd.0 0 log_to_monitors true 2024-04-25T19:02:22.434 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T19:02:22.433+0000 7ff637f57640 -1 osd.3 0 log_to_monitors true 2024-04-25T19:02:22.440 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T19:02:22.438+0000 7ffab3f55640 -1 osd.4 0 log_to_monitors true 2024-04-25T19:02:22.584 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T19:02:22.583+0000 7f0194a86640 -1 osd.1 0 log_to_monitors true 2024-04-25T19:02:22.585 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T19:02:22.582+0000 7fe2f1a0c640 -1 osd.5 0 log_to_monitors true 2024-04-25T19:02:22.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:02:22.592 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:02:22.641 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T19:02:22.639+0000 7fbbf0ffc640 -1 osd.7 0 log_to_monitors true 2024-04-25T19:02:24.294 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T19:02:24.292+0000 7f46e9600640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T19:02:24.326 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T19:02:24.323+0000 7f46dec00640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:25.882 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T19:02:25.880+0000 7ffaaee00640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T19:02:25.894 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T19:02:25.891+0000 7ffaa4e00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:25.968 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T19:02:25.965+0000 7fe2ec200640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T19:02:25.975 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T19:02:25.973+0000 7fe2e2200640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:25.995 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T19:02:25.993+0000 7fbbebe00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T19:02:26.006 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T19:02:26.004+0000 7fbbe1e00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:26.387 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:26.457 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T19:02:26.455+0000 7f5dd7800640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T19:02:26.476 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-25T19:02:26.475+0000 7f5dcd800640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:26.508 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T19:02:26.506+0000 7ff632e00640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T19:02:26.510 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T19:02:26.509+0000 7f0190400640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T19:02:26.516 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-25T19:02:26.514+0000 7ff628e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:26.526 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-25T19:02:26.524+0000 7f0185a00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:26.563 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T19:02:26.561+0000 7fde31c00640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T19:02:26.570 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-25T19:02:26.568+0000 7fde27200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:02:26.657 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.655+0000 7fdf2d600640 1 Processor -- start 2024-04-25T19:02:26.657 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.656+0000 7fdf2d600640 1 -- start start 2024-04-25T19:02:26.659 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.657+0000 7fdf2d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf28168d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.659 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.657+0000 7fdf2d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf280d0900 con 0x7fdf280d0ff0 2024-04-25T19:02:26.659 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.657+0000 7fdf2d600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf2815e720 con 0x7fdf2815e1d0 2024-04-25T19:02:26.659 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.657+0000 7fdf2d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2815ea00 con 0x7fdf2806d580 2024-04-25T19:02:26.659 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf26000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdf280d0ff0 0x7fdf280d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48946/0 (socket says 172.21.15.123:48946) 2024-04-25T19:02:26.660 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf26000640 1 -- 172.21.15.123:0/3651516959 learned_addr learned my addr 172.21.15.123:0/3651516959 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:26.662 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818673078 0 0) 0x7fdf280d0900 con 0x7fdf280d0ff0 2024-04-25T19:02:26.662 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf27400640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf28168d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.662 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf0c003520 con 0x7fdf280d0ff0 2024-04-25T19:02:26.662 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1021487289 0 0) 0x7fdf0c003520 con 0x7fdf280d0ff0 2024-04-25T19:02:26.662 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 msgr2=0x7fdf28168d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf28168d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 >> v1:172.21.15.146:6789/0 conn(0x7fdf2815e1d0 legacy=0x7fdf280cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2815ece0 con 0x7fdf280d0ff0 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.658+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fdf14001670 con 0x7fdf280d0ff0 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf27400640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf28168d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 >> v1:172.21.15.123:6789/0 conn(0x7fdf280d0ff0 legacy=0x7fdf280d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0c0039e0 0x7fdf0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf0c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf0c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2815ea00 con 0x7fdf0c004800 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c00bb80 con 0x7fdf0c0039e0 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c00bd80 con 0x7fdf2806d580 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf0c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.663 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26a00640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0c0039e0 0x7fdf0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf27400640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf0c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 -- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 msgr2=0x7fdf0c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf2806d580 0x7fdf0c005290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 -- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0c0039e0 msgr2=0x7fdf0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf0c0039e0 0x7fdf0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.659+0000 7fdf26000640 1 -- 172.21.15.123:0/3651516959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c003520 con 0x7fdf0c004800 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.660+0000 7fdf26000640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf0c004c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c210 tx=0x7fdf0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e191d456d3d35e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.660+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14007d30 con 0x7fdf0c004800 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.660+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf14005ad0 con 0x7fdf0c004800 2024-04-25T19:02:26.664 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.660+0000 7fdf25600640 1 -- 172.21.15.123:0/3651516959 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14007070 con 0x7fdf0c004800 2024-04-25T19:02:26.665 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 -- 172.21.15.123:0/3651516959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 msgr2=0x7fdf0c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.665 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 --2- 172.21.15.123:0/3651516959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf0c004c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c210 tx=0x7fdf0c005830 comp rx=0 tx=0).stop 2024-04-25T19:02:26.665 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf27400640 1 -- 172.21.15.123:0/3651516959 reap_dead start 2024-04-25T19:02:26.665 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 -- 172.21.15.123:0/3651516959 shutdown_connections 2024-04-25T19:02:26.665 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 -- 172.21.15.123:0/3651516959 >> 172.21.15.123:0/3651516959 conn(0x7fdf280ce190 msgr2=0x7fdf280ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.666 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 -- 172.21.15.123:0/3651516959 shutdown_connections 2024-04-25T19:02:26.666 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.661+0000 7fdf2d600640 1 -- 172.21.15.123:0/3651516959 wait complete. 2024-04-25T19:02:26.666 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.664+0000 7fdf2d600640 1 Processor -- start 2024-04-25T19:02:26.666 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.664+0000 7fdf2d600640 1 -- start start 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf28154080 0x7fdf28148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf281545d0 0x7fdf2815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28166d70 con 0x7fdf0c004800 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf280d1400 con 0x7fdf281545d0 2024-04-25T19:02:26.667 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.665+0000 7fdf2d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf28166ef0 con 0x7fdf28154080 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33126/0 (socket says 172.21.15.123:33126) 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 -- 172.21.15.123:0/2756084061 learned_addr learned my addr 172.21.15.123:0/2756084061 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf27400640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf281545d0 0x7fdf2815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 -- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf28154080 msgr2=0x7fdf28148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf28154080 0x7fdf28148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 -- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf281545d0 msgr2=0x7fdf2815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf281545d0 0x7fdf2815a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 -- 172.21.15.123:0/2756084061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf28155a70 con 0x7fdf0c004800 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.666+0000 7fdf26a00640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e580 tx=0x7fdf1800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.667+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180052e0 con 0x7fdf0c004800 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.667+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf18018070 con 0x7fdf0c004800 2024-04-25T19:02:26.669 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.667+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18003810 con 0x7fdf0c004800 2024-04-25T19:02:26.670 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.667+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf28155790 con 0x7fdf0c004800 2024-04-25T19:02:26.670 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.667+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf280b1f10 con 0x7fdf0c004800 2024-04-25T19:02:26.670 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.668+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf28076a40 con 0x7fdf0c004800 2024-04-25T19:02:26.670 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.669+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7fdf180039b0 con 0x7fdf0c004800 2024-04-25T19:02:26.671 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.670+0000 7fdf1ea00640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 0x7fdf08076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.672 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.670+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7fdf18094200 con 0x7fdf0c004800 2024-04-25T19:02:26.672 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.670+0000 7fdf26000640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 0x7fdf08076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.672 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.671+0000 7fdf26000640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 0x7fdf08076960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf14033000 tx=0x7fdf14009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.677 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.675+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf1805f6f0 con 0x7fdf0c004800 2024-04-25T19:02:26.860 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.859+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf280ce8b0 con 0x7fdf0c004800 2024-04-25T19:02:26.861 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.860+0000 7fdf1ea00640 1 -- 172.21.15.123:0/2756084061 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7555 (secure 0 0 0) 0x7fdf1805eff0 con 0x7fdf0c004800 2024-04-25T19:02:26.862 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-25T19:02:26.862 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":14,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:25.337895+0000","last_up_change":"2024-04-25T19:02:25.337895+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 msgr2=0x7fdf08076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 0x7fdf08076960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf14033000 tx=0x7fdf14009030 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 msgr2=0x7fdf280b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e580 tx=0x7fdf1800ea40 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 shutdown_connections 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf08074450 0x7fdf08076960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf281545d0 0x7fdf2815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf28154080 0x7fdf28148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 --2- 172.21.15.123:0/2756084061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf0c004800 0x7fdf280b2710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.864 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 >> 172.21.15.123:0/2756084061 conn(0x7fdf280ce190 msgr2=0x7fdf2815f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.865 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.862+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 shutdown_connections 2024-04-25T19:02:26.865 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:26.863+0000 7fdf2d600640 1 -- 172.21.15.123:0/2756084061 wait complete. 2024-04-25T19:02:26.884 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T19:02:33.691 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:33.948 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.946+0000 7f7bf3200640 1 Processor -- start 2024-04-25T19:02:33.948 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.946+0000 7f7bf3200640 1 -- start start 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.948+0000 7f7bf3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bec164200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.948+0000 7f7bf3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec164740 con 0x7f7bec15a6b0 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.948+0000 7f7bf3200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec06d020 con 0x7f7bec168170 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.948+0000 7f7bf3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec06dc10 con 0x7f7bec163df0 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.949+0000 7f7bebe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7bec15a6b0 0x7f7bec06f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48958/0 (socket says 172.21.15.123:48958) 2024-04-25T19:02:33.950 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.949+0000 7f7bebe00640 1 -- 172.21.15.123:0/3336129786 learned_addr learned my addr 172.21.15.123:0/3336129786 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:33.952 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.949+0000 7f7beb400640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bec164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.949+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180084440 0 0) 0x7f7bdc001490 con 0x7f7bec15a6b0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.949+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bd0003520 con 0x7f7bec15a6b0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2742831315 0 0) 0x7f7bd0003520 con 0x7f7bec15a6b0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 msgr2=0x7f7bec164200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bec164200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 >> v1:172.21.15.146:6789/0 conn(0x7f7bec168170 legacy=0x7f7bec16a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec06d2e0 con 0x7f7bec15a6b0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f7bdc001bc0 con 0x7f7bec15a6b0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 >> v1:172.21.15.123:6789/0 conn(0x7f7bec15a6b0 legacy=0x7f7bec06f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd00039e0 0x7f7bd0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd0004d40 0x7f7bd0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec06dc10 con 0x7f7bd00039e0 2024-04-25T19:02:33.953 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000baf0 con 0x7f7bd0004d40 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd000bcf0 con 0x7f7bec163df0 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7bebe00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd00039e0 0x7f7bd0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.950+0000 7f7bf1200640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd0004d40 0x7f7bd0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7beb400640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7bebe00640 1 -- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 msgr2=0x7f7bd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7bebe00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec163df0 0x7f7bd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7bebe00640 1 -- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd0004d40 msgr2=0x7f7bd0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7bebe00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd0004d40 0x7f7bd0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:33.954 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.951+0000 7f7bebe00640 1 -- 172.21.15.123:0/3336129786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd0003520 con 0x7f7bd00039e0 2024-04-25T19:02:33.955 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.952+0000 7f7bebe00640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd00039e0 0x7f7bd0003e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00aa50 tx=0x7f7bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11edac12cdc1ca89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:33.955 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.952+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00a670 con 0x7f7bd00039e0 2024-04-25T19:02:33.955 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.952+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bdc02f070 con 0x7f7bd00039e0 2024-04-25T19:02:33.955 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.952+0000 7f7beaa00640 1 -- 172.21.15.123:0/3336129786 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc02be00 con 0x7f7bd00039e0 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 -- 172.21.15.123:0/3336129786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd00039e0 msgr2=0x7f7bd0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 --2- 172.21.15.123:0/3336129786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd00039e0 0x7f7bd0003e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00aa50 tx=0x7f7bd0005830 comp rx=0 tx=0).stop 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf1200640 1 -- 172.21.15.123:0/3336129786 reap_dead start 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 -- 172.21.15.123:0/3336129786 shutdown_connections 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 -- 172.21.15.123:0/3336129786 >> 172.21.15.123:0/3336129786 conn(0x7f7bec0ce190 msgr2=0x7f7bec0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 -- 172.21.15.123:0/3336129786 shutdown_connections 2024-04-25T19:02:33.956 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.953+0000 7f7bf3200640 1 -- 172.21.15.123:0/3336129786 wait complete. 2024-04-25T19:02:33.957 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.956+0000 7f7bf3200640 1 Processor -- start 2024-04-25T19:02:33.957 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.956+0000 7f7bf3200640 1 -- start start 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd00039e0 0x7f7bec160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec164e10 0x7f7bec165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec164b40 con 0x7f7bec166100 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec06d8a0 con 0x7f7bd00039e0 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.957+0000 7f7bf3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec06f800 con 0x7f7bec164e10 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33184/0 (socket says 172.21.15.123:33184) 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 -- 172.21.15.123:0/1220117072 learned_addr learned my addr 172.21.15.123:0/1220117072 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7bebe00640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd00039e0 0x7f7bec160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.960 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 -- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec164e10 msgr2=0x7f7bec165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:33.961 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec164e10 0x7f7bec165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:33.961 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 -- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd00039e0 msgr2=0x7f7bec160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:33.961 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd00039e0 0x7f7bec160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:33.961 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 -- 172.21.15.123:0/1220117072 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec0ae7e0 con 0x7f7bec166100 2024-04-25T19:02:33.961 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.958+0000 7f7beb400640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8011630 tx=0x7f7bd8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:33.963 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.962+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800f920 con 0x7f7bec166100 2024-04-25T19:02:33.964 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.962+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec167060 con 0x7f7bec166100 2024-04-25T19:02:33.964 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.962+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec0aed80 con 0x7f7bec166100 2024-04-25T19:02:33.964 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.962+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd800fac0 con 0x7f7bec166100 2024-04-25T19:02:33.964 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.962+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800fe70 con 0x7f7bec166100 2024-04-25T19:02:33.964 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.963+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc4005290 con 0x7f7bec166100 2024-04-25T19:02:33.965 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.964+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7bd8015460 con 0x7f7bec166100 2024-04-25T19:02:33.967 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.965+0000 7f7be3e00640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:33.967 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.965+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7bd8095930 con 0x7f7bec166100 2024-04-25T19:02:33.967 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.965+0000 7f7bebe00640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:33.968 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.966+0000 7f7bebe00640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00aa20 tx=0x7f7bdc038040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:33.972 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:33.971+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bd8019030 con 0x7f7bec166100 2024-04-25T19:02:34.173 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.171+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bc4005850 con 0x7f7bec166100 2024-04-25T19:02:34.175 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.174+0000 7f7be3e00640 1 -- 172.21.15.123:0/1220117072 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7f7bd80603e0 con 0x7f7bec166100 2024-04-25T19:02:34.176 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-25T19:02:34.176 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":18,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:29.917060+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 msgr2=0x7f7bcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00aa20 tx=0x7f7bdc038040 comp rx=0 tx=0).stop 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 msgr2=0x7f7bec165560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8011630 tx=0x7f7bd8011af0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 shutdown_connections 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.178 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7bec164e10 0x7f7bec165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.179 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bec166100 0x7f7bec165560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.179 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 --2- 172.21.15.123:0/1220117072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bd00039e0 0x7f7bec160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.179 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 >> 172.21.15.123:0/1220117072 conn(0x7f7bec0ce190 msgr2=0x7f7bec162f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.179 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 shutdown_connections 2024-04-25T19:02:34.179 INFO:teuthology.misc.health.smithi123.stderr:2024-04-25T19:02:34.176+0000 7f7bf3200640 1 -- 172.21.15.123:0/1220117072 wait complete. 2024-04-25T19:02:34.198 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T19:02:34.198 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T19:02:34.199 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T19:02:34.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T19:02:34.206 INFO:tasks.ceph.mds.a:Started 2024-04-25T19:02:34.206 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T19:02:34.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T19:02:34.213 INFO:tasks.ceph.mds.c:Started 2024-04-25T19:02:34.213 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T19:02:34.213 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T19:02:34.221 INFO:tasks.ceph.mds.b:Started 2024-04-25T19:02:34.221 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T19:02:34.221 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T19:02:34.228 INFO:tasks.ceph.mds.d:Started 2024-04-25T19:02:34.229 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T19:02:34.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:34.422 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-25T19:02:34.422 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-25T19:02:34.454 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-04-25T19:02:34.456 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-04-25T19:02:34.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.472+0000 7f7d3c600640 1 Processor -- start 2024-04-25T19:02:34.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.472+0000 7f7d3c600640 1 -- start start 2024-04-25T19:02:34.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d3c600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7d340cfd10 0x7f7d340b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d3c600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d340b3c30 con 0x7f7d3406f500 2024-04-25T19:02:34.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d3c600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d340b3f10 con 0x7f7d340d22e0 2024-04-25T19:02:34.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d3c600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7d340b39e0 con 0x7f7d340cfd10 2024-04-25T19:02:34.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d39200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7d3406f500 0x7f7d340d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48966/0 (socket says 172.21.15.123:48966) 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d39200640 1 -- 172.21.15.123:0/2115735106 learned_addr learned my addr 172.21.15.123:0/2115735106 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.474+0000 7f7d39c00640 1 --2- 172.21.15.123:0/2115735106 >> v2:172.21.15.146:3300/0 conn(0x7f7d340cfd10 0x7f7d340b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431235153 0 0) 0x7f7d340b3c30 con 0x7f7d3406f500 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d1c003520 con 0x7f7d3406f500 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037769832 0 0) 0x7f7d340b3f10 con 0x7f7d340d22e0 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d340b3c30 con 0x7f7d340d22e0 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d39c00640 1 -- 172.21.15.123:0/2115735106 >> v1:172.21.15.146:6789/0 conn(0x7f7d340d22e0 legacy=0x7f7d340b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d39c00640 1 -- 172.21.15.123:0/2115735106 >> v1:172.21.15.123:6789/0 conn(0x7f7d3406f500 legacy=0x7f7d340d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.475+0000 7f7d39c00640 1 -- 172.21.15.123:0/2115735106 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340b41f0 con 0x7f7d340cfd10 2024-04-25T19:02:34.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.476+0000 7f7d39c00640 1 --2- 172.21.15.123:0/2115735106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340cfd10 0x7f7d340b4cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720e9f9473e4dbaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.476+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28037070 con 0x7f7d340cfd10 2024-04-25T19:02:34.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.476+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d28032440 con 0x7f7d340cfd10 2024-04-25T19:02:34.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.476+0000 7f7d33e00640 1 -- 172.21.15.123:0/2115735106 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28032810 con 0x7f7d340cfd10 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 -- 172.21.15.123:0/2115735106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340cfd10 msgr2=0x7f7d340b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 --2- 172.21.15.123:0/2115735106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340cfd10 0x7f7d340b4cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 -- 172.21.15.123:0/2115735106 shutdown_connections 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 --2- 172.21.15.123:0/2115735106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340cfd10 0x7f7d340b4cd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 -- 172.21.15.123:0/2115735106 >> 172.21.15.123:0/2115735106 conn(0x7f7d3409b4a0 msgr2=0x7f7d3409b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 -- 172.21.15.123:0/2115735106 shutdown_connections 2024-04-25T19:02:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.477+0000 7f7d3c600640 1 -- 172.21.15.123:0/2115735106 wait complete. 2024-04-25T19:02:34.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.479+0000 7f7d3c600640 1 Processor -- start 2024-04-25T19:02:34.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.479+0000 7f7d3c600640 1 -- start start 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d3406f500 0x7f7d34162970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 0x7f7d3415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3406f910 con 0x7f7d340cfd10 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340b47b0 con 0x7f7d340d22e0 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3c600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7d34164a80 con 0x7f7d3406f500 2024-04-25T19:02:34.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d39200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d39200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33208/0 (socket says 172.21.15.123:33208) 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d39200640 1 -- 172.21.15.123:0/3122502516 learned_addr learned my addr 172.21.15.123:0/3122502516 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d39c00640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d3406f500 0x7f7d34162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.481+0000 7f7d3a600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 0x7f7d3415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 -- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d3406f500 msgr2=0x7f7d34162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d3406f500 0x7f7d34162970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 -- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 msgr2=0x7f7d3415ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 0x7f7d3415ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 -- 172.21.15.123:0/3122502516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d34161920 con 0x7f7d340cfd10 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d3a600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 0x7f7d3415ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d39200640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c3c0 tx=0x7f7d2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20017070 con 0x7f7d340cfd10 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d20005da0 con 0x7f7d340cfd10 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341f6bf0 con 0x7f7d340cfd10 2024-04-25T19:02:34.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20007310 con 0x7f7d340cfd10 2024-04-25T19:02:34.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.482+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341f71b0 con 0x7f7d340cfd10 2024-04-25T19:02:34.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.484+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34161df0 con 0x7f7d340cfd10 2024-04-25T19:02:34.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.485+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d2001a070 con 0x7f7d340cfd10 2024-04-25T19:02:34.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.486+0000 7f7d31600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 0x7f7d140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.486+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7d20094310 con 0x7f7d340cfd10 2024-04-25T19:02:34.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.486+0000 7f7d39c00640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 0x7f7d140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.487+0000 7f7d39c00640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 0x7f7d140769c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d28034000 tx=0x7f7d2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.491+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d2005e720 con 0x7f7d340cfd10 2024-04-25T19:02:34.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.678+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d340af6e0 con 0x7f7d340cfd10 2024-04-25T19:02:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.680+0000 7f7d31600640 1 -- 172.21.15.123:0/3122502516 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7f7d20062a20 con 0x7f7d340cfd10 2024-04-25T19:02:34.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:34.682 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:29.917060+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 msgr2=0x7f7d140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 0x7f7d140769c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d28034000 tx=0x7f7d2802e000 comp rx=0 tx=0).stop 2024-04-25T19:02:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 msgr2=0x7f7d341632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c3c0 tx=0x7f7d2000c880 comp rx=0 tx=0).stop 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 shutdown_connections 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d140744b0 0x7f7d140769c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d340d22e0 0x7f7d3415ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d340cfd10 0x7f7d341632b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 --2- 172.21.15.123:0/3122502516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d3406f500 0x7f7d34162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 >> 172.21.15.123:0/3122502516 conn(0x7f7d3409b4a0 msgr2=0x7f7d340d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 shutdown_connections 2024-04-25T19:02:34.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.683+0000 7f7d3c600640 1 -- 172.21.15.123:0/3122502516 wait complete. 2024-04-25T19:02:34.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:02:34.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:02:34.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.981+0000 7f4dea200640 1 Processor -- start 2024-04-25T19:02:34.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.981+0000 7f4dea200640 1 -- start start 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4dea200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de415a850 0x7f4de409b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4dea200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4de409b9e0 0x7f4de40b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4dea200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de415ac60 con 0x7f4de4160770 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4dea200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4de409bdf0 con 0x7f4de409b9e0 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4dea200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4de4160ce0 con 0x7f4de415a850 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.983+0000 7f4de3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4de409b9e0 0x7f4de40b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4de409b9e0 0x7f4de40b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33212/0 (socket says 172.21.15.123:33212) 2024-04-25T19:02:34.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 -- 172.21.15.123:0/446742634 learned_addr learned my addr 172.21.15.123:0/446742634 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:34.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de2a00640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de415a850 0x7f4de409b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 -- 172.21.15.123:0/446742634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de415a850 msgr2=0x7f4de409b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de415a850 0x7f4de409b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 -- 172.21.15.123:0/446742634 >> v1:172.21.15.146:6789/0 conn(0x7f4de4160770 legacy=0x7f4de406f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.984+0000 7f4de3e00640 1 -- 172.21.15.123:0/446742634 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de4160fc0 con 0x7f4de409b9e0 2024-04-25T19:02:34.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.985+0000 7f4de3e00640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de409b9e0 0x7f4de40b3070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009e50 tx=0x7f4ddc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7962c44fa10373e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.985+0000 7f4de2000640 1 -- 172.21.15.123:0/446742634 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc036070 con 0x7f4de409b9e0 2024-04-25T19:02:34.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.985+0000 7f4de2000640 1 -- 172.21.15.123:0/446742634 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ddc031600 con 0x7f4de409b9e0 2024-04-25T19:02:34.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.985+0000 7f4de2000640 1 -- 172.21.15.123:0/446742634 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0319d0 con 0x7f4de409b9e0 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 -- 172.21.15.123:0/446742634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de409b9e0 msgr2=0x7f4de40b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de409b9e0 0x7f4de40b3070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009e50 tx=0x7f4ddc02a6c0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 -- 172.21.15.123:0/446742634 shutdown_connections 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de409b9e0 0x7f4de40b3070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 --2- 172.21.15.123:0/446742634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de415a850 0x7f4de409b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 -- 172.21.15.123:0/446742634 >> 172.21.15.123:0/446742634 conn(0x7f4de40b7680 msgr2=0x7f4de40b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 -- 172.21.15.123:0/446742634 shutdown_connections 2024-04-25T19:02:34.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.986+0000 7f4dea200640 1 -- 172.21.15.123:0/446742634 wait complete. 2024-04-25T19:02:34.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.988+0000 7f4dea200640 1 Processor -- start 2024-04-25T19:02:34.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.989+0000 7f4dea200640 1 -- start start 2024-04-25T19:02:34.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de409b9e0 0x7f4de409d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 0x7f4de406df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4de40b35b0 con 0x7f4de415a850 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4161580 con 0x7f4de4160770 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4dea200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4de40b3730 con 0x7f4de409b9e0 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33226/0 (socket says 172.21.15.123:33226) 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.990+0000 7f4de2a00640 1 -- 172.21.15.123:0/1219229501 learned_addr learned my addr 172.21.15.123:0/1219229501 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de3400640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de409b9e0 0x7f4de409d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de3e00640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 0x7f4de406df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 -- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de409b9e0 msgr2=0x7f4de409d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de409b9e0 0x7f4de409d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 -- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 msgr2=0x7f4de406df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 0x7f4de406df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 -- 172.21.15.123:0/1219229501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41faad0 con 0x7f4de415a850 2024-04-25T19:02:34.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de3e00640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 0x7f4de406df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:34.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4de2a00640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000d620 tx=0x7f4dd000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00125f0 con 0x7f4de415a850 2024-04-25T19:02:34.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd0019070 con 0x7f4de415a850 2024-04-25T19:02:34.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00129d0 con 0x7f4de415a850 2024-04-25T19:02:34.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.991+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de40d31e0 con 0x7f4de415a850 2024-04-25T19:02:34.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.992+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de4152630 con 0x7f4de415a850 2024-04-25T19:02:34.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.993+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de4076a40 con 0x7f4de415a850 2024-04-25T19:02:34.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.994+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4dd0012b70 con 0x7f4de415a850 2024-04-25T19:02:34.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.995+0000 7f4ddb400640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 0x7f4dc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.995+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f4dd00953b0 con 0x7f4de415a850 2024-04-25T19:02:34.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.995+0000 7f4de3400640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 0x7f4dc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.996+0000 7f4de3400640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 0x7f4dc40769c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4006fd0 tx=0x7f4dd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:35.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:34.999+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dd005fdf0 con 0x7f4de415a850 2024-04-25T19:02:35.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.198+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4de40b2600 con 0x7f4de415a850 2024-04-25T19:02:35.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.199+0000 7f4ddb400640 1 -- 172.21.15.123:0/1219229501 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f4dd0066e20 con 0x7f4de415a850 2024-04-25T19:02:35.200 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:02:35.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.201+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 msgr2=0x7f4dc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.201+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 0x7f4dc40769c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4006fd0 tx=0x7f4dd4008040 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.201+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 msgr2=0x7f4de4151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.201+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000d620 tx=0x7f4dd000dae0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 shutdown_connections 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dc40744b0 0x7f4dc40769c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4de4160770 0x7f4de406df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4de415a850 0x7f4de4151ac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 --2- 172.21.15.123:0/1219229501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4de409b9e0 0x7f4de409d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 >> 172.21.15.123:0/1219229501 conn(0x7f4de40b7680 msgr2=0x7f4de4161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 shutdown_connections 2024-04-25T19:02:35.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.202+0000 7f4dea200640 1 -- 172.21.15.123:0/1219229501 wait complete. 2024-04-25T19:02:35.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:02:35.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.487+0000 7f32d8a00640 1 Processor -- start 2024-04-25T19:02:35.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.487+0000 7f32d8a00640 1 -- start start 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d8a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32d413fbf0 0x7f32d406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d8a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 0x7f32d40b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d8a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d406ded0 con 0x7f32d406dac0 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d8a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f32d4143050 con 0x7f32d413fbf0 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d8a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32d40b5c70 con 0x7f32d4142c40 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d2800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32d413fbf0 0x7f32d406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d1e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f32d406dac0 0x7f32d4142530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57770/0 (socket says 172.21.15.123:57770) 2024-04-25T19:02:35.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.489+0000 7f32d1e00640 1 -- 172.21.15.123:0/4241159145 learned_addr learned my addr 172.21.15.123:0/4241159145 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d3200640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 0x7f32d40b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d2800640 1 -- 172.21.15.123:0/4241159145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 msgr2=0x7f32d40b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d2800640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 0x7f32d40b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d2800640 1 -- 172.21.15.123:0/4241159145 >> v1:172.21.15.123:6789/0 conn(0x7f32d406dac0 legacy=0x7f32d4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d2800640 1 -- 172.21.15.123:0/4241159145 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d40b5f50 con 0x7f32d413fbf0 2024-04-25T19:02:35.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.490+0000 7f32d3200640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 0x7f32d40b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.491+0000 7f32d2800640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d406d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c40097a0 tx=0x7f32c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dfc6f90383a12f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.491+0000 7f32d1400640 1 -- 172.21.15.123:0/4241159145 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4037070 con 0x7f32d413fbf0 2024-04-25T19:02:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.491+0000 7f32d1400640 1 -- 172.21.15.123:0/4241159145 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c4032440 con 0x7f32d413fbf0 2024-04-25T19:02:35.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.491+0000 7f32d1400640 1 -- 172.21.15.123:0/4241159145 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4032810 con 0x7f32d413fbf0 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 -- 172.21.15.123:0/4241159145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 msgr2=0x7f32d406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d406d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c40097a0 tx=0x7f32c402a180 comp rx=0 tx=0).stop 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 -- 172.21.15.123:0/4241159145 shutdown_connections 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d4142c40 0x7f32d40b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 --2- 172.21.15.123:0/4241159145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d406d580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 -- 172.21.15.123:0/4241159145 >> 172.21.15.123:0/4241159145 conn(0x7f32d4073200 msgr2=0x7f32d4073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 -- 172.21.15.123:0/4241159145 shutdown_connections 2024-04-25T19:02:35.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.492+0000 7f32d8a00640 1 -- 172.21.15.123:0/4241159145 wait complete. 2024-04-25T19:02:35.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.494+0000 7f32d8a00640 1 Processor -- start 2024-04-25T19:02:35.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.495+0000 7f32d8a00640 1 -- start start 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d406dac0 0x7f32d4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32d4142c40 0x7f32d40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32d40b6a30 con 0x7f32d413fbf0 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32d40b6510 con 0x7f32d4142c40 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d8a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32d40b4b80 con 0x7f32d406dac0 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.496+0000 7f32d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39968/0 (socket says 172.21.15.123:39968) 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 -- 172.21.15.123:0/2466280796 learned_addr learned my addr 172.21.15.123:0/2466280796 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d2800640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d406dac0 0x7f32d4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d3200640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32d4142c40 0x7f32d40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 -- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d406dac0 msgr2=0x7f32d4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d406dac0 0x7f32d4156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 -- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32d4142c40 msgr2=0x7f32d40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32d4142c40 0x7f32d40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 -- 172.21.15.123:0/2466280796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d413ac20 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32d1e00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c3c0 tx=0x7f32c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.497+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0017070 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.498+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c0005ce0 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.498+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d413a660 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.498+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00072f0 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.498+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41e6660 con 0x7f32d413fbf0 2024-04-25T19:02:35.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.499+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d4050740 con 0x7f32d413fbf0 2024-04-25T19:02:35.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.500+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f32c001a070 con 0x7f32d413fbf0 2024-04-25T19:02:35.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.501+0000 7f32caa00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 0x7f32b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:35.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.501+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f32c00942f0 con 0x7f32d413fbf0 2024-04-25T19:02:35.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.501+0000 7f32d2800640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 0x7f32b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:35.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.502+0000 7f32d2800640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 0x7f32b40769f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32c4034000 tx=0x7f32c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:35.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.505+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32d406ded0 con 0x7f32d413fbf0 2024-04-25T19:02:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.744+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32d40cfb70 con 0x7f32d413fbf0 2024-04-25T19:02:35.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.745+0000 7f32caa00640 1 -- 172.21.15.123:0/2466280796 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f32c005ed30 con 0x7f32d413fbf0 2024-04-25T19:02:35.747 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:35.748 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T19:02:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-04-25T19:02:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 msgr2=0x7f32b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 0x7f32b40769f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32c4034000 tx=0x7f32c402e000 comp rx=0 tx=0).stop 2024-04-25T19:02:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 msgr2=0x7f32d4139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:35.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c3c0 tx=0x7f32c000c880 comp rx=0 tx=0).stop 2024-04-25T19:02:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 shutdown_connections 2024-04-25T19:02:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f32b40744e0 0x7f32b40769f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32d4142c40 0x7f32d40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d413fbf0 0x7f32d4139cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 --2- 172.21.15.123:0/2466280796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32d406dac0 0x7f32d4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 >> 172.21.15.123:0/2466280796 conn(0x7f32d4073200 msgr2=0x7f32d413c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 shutdown_connections 2024-04-25T19:02:35.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:35.748+0000 7f32d8a00640 1 -- 172.21.15.123:0/2466280796 wait complete. 2024-04-25T19:02:35.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:36.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.027+0000 7ff8f5a00640 1 Processor -- start 2024-04-25T19:02:36.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.027+0000 7ff8f5a00640 1 -- start start 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.029+0000 7ff8f5a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8f00b57f0 0x7ff8f00b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.029+0000 7ff8f5a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f00b6140 con 0x7ff8f015df00 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.029+0000 7ff8f5a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f00b4750 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.029+0000 7ff8f5a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f015e310 con 0x7ff8f00b57f0 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8e7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8f00b57f0 0x7ff8f00b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8eea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff8f015df00 0x7ff8f006f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57780/0 (socket says 172.21.15.123:57780) 2024-04-25T19:02:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8eea00640 1 -- 172.21.15.123:0/3724417823 learned_addr learned my addr 172.21.15.123:0/3724417823 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8e7e00640 1 -- 172.21.15.123:0/3724417823 >> v1:172.21.15.146:6789/0 conn(0x7ff8f00ae7e0 legacy=0x7ff8f00aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8e7e00640 1 -- 172.21.15.123:0/3724417823 >> v1:172.21.15.123:6789/0 conn(0x7ff8f015df00 legacy=0x7ff8f006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.030+0000 7ff8e7e00640 1 -- 172.21.15.123:0/3724417823 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00b4a30 con 0x7ff8f00b57f0 2024-04-25T19:02:36.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.031+0000 7ff8e7e00640 1 --2- 172.21.15.123:0/3724417823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00b57f0 0x7ff8f00b5c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80097a0 tx=0x7ff8d802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4d6ba37678243f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.031+0000 7ff8ee000640 1 -- 172.21.15.123:0/3724417823 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8039070 con 0x7ff8f00b57f0 2024-04-25T19:02:36.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.031+0000 7ff8ee000640 1 -- 172.21.15.123:0/3724417823 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d8034640 con 0x7ff8f00b57f0 2024-04-25T19:02:36.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.031+0000 7ff8ee000640 1 -- 172.21.15.123:0/3724417823 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d802ce60 con 0x7ff8f00b57f0 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.032+0000 7ff8f5a00640 1 -- 172.21.15.123:0/3724417823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00b57f0 msgr2=0x7ff8f00b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.032+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/3724417823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00b57f0 0x7ff8f00b5c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80097a0 tx=0x7ff8d802c350 comp rx=0 tx=0).stop 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.032+0000 7ff8f5a00640 1 -- 172.21.15.123:0/3724417823 shutdown_connections 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.032+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/3724417823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00b57f0 0x7ff8f00b5c00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.033+0000 7ff8f5a00640 1 -- 172.21.15.123:0/3724417823 >> 172.21.15.123:0/3724417823 conn(0x7ff8f00d1ae0 msgr2=0x7ff8f00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:36.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.033+0000 7ff8f5a00640 1 -- 172.21.15.123:0/3724417823 shutdown_connections 2024-04-25T19:02:36.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.033+0000 7ff8f5a00640 1 -- 172.21.15.123:0/3724417823 wait complete. 2024-04-25T19:02:36.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.035+0000 7ff8f5a00640 1 Processor -- start 2024-04-25T19:02:36.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.035+0000 7ff8f5a00640 1 -- start start 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8f00b57f0 0x7ff8f01f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8f015df00 0x7ff8f01f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00b4ff0 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00b5360 con 0x7ff8f015df00 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8f5a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f006f950 con 0x7ff8f00b57f0 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8eea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8eea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39982/0 (socket says 172.21.15.123:39982) 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8eea00640 1 -- 172.21.15.123:0/1343527174 learned_addr learned my addr 172.21.15.123:0/1343527174 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.037+0000 7ff8ef400640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8f015df00 0x7ff8f01f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8e7e00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8f00b57f0 0x7ff8f01f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 -- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8f00b57f0 msgr2=0x7ff8f01f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8f00b57f0 0x7ff8f01f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 -- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8f015df00 msgr2=0x7ff8f01f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8f015df00 0x7ff8f01f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 -- 172.21.15.123:0/1343527174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00cabb0 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8eea00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c3c0 tx=0x7ff8e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0017070 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e0005ce0 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0007310 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f00cae90 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.038+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f00cb970 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.040+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f0076a40 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.042+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8e001a070 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.044+0000 7ff8e6a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 0x7ff8cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.044+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff8e0094240 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.044+0000 7ff8e7e00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 0x7ff8cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.045+0000 7ff8e7e00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 0x7ff8cc0769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8036000 tx=0x7ff8d8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.049+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8e005ed90 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.252+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8f0070ec0 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.255+0000 7ff8e6a00640 1 -- 172.21.15.123:0/1343527174 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7ff8e005e690 con 0x7ff8f00ae7e0 2024-04-25T19:02:36.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:36.257 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:29.917060+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 msgr2=0x7ff8cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 0x7ff8cc0769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8036000 tx=0x7ff8d8030000 comp rx=0 tx=0).stop 2024-04-25T19:02:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 msgr2=0x7ff8f00cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c3c0 tx=0x7ff8e000c880 comp rx=0 tx=0).stop 2024-04-25T19:02:36.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 shutdown_connections 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff8cc0744e0 0x7ff8cc0769f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8f015df00 0x7ff8f01f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8f00b57f0 0x7ff8f01f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 --2- 172.21.15.123:0/1343527174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f00ae7e0 0x7ff8f00cd5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 >> 172.21.15.123:0/1343527174 conn(0x7ff8f00d1ae0 msgr2=0x7ff8f015b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 shutdown_connections 2024-04-25T19:02:36.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.257+0000 7ff8f5a00640 1 -- 172.21.15.123:0/1343527174 wait complete. 2024-04-25T19:02:36.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:02:36.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:02:36.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.549+0000 7f0f13e00640 1 Processor -- start 2024-04-25T19:02:36.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.549+0000 7f0f13e00640 1 -- start start 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f13e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f1406dc90 0x7f0f14167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f13e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f140d05f0 con 0x7f0f14143040 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f13e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f140d08d0 con 0x7f0f140cfff0 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f13e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140d1200 con 0x7f0f1406dc90 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f0a000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0f14143040 0x7f0f1406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57782/0 (socket says 172.21.15.123:57782) 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f0a000640 1 -- 172.21.15.123:0/1195566816 learned_addr learned my addr 172.21.15.123:0/1195566816 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:36.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.552+0000 7f0f13400640 1 --2- 172.21.15.123:0/1195566816 >> v2:172.21.15.146:3300/0 conn(0x7f0f1406dc90 0x7f0f14167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344445087 0 0) 0x7f0f140d05f0 con 0x7f0f14143040 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ef8003520 con 0x7f0f14143040 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2152044241 0 0) 0x7f0ef8003520 con 0x7f0f14143040 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 >> v2:172.21.15.146:3300/0 conn(0x7f0f1406dc90 msgr2=0x7f0f14167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 --2- 172.21.15.123:0/1195566816 >> v2:172.21.15.146:3300/0 conn(0x7f0f1406dc90 0x7f0f14167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 >> v1:172.21.15.146:6789/0 conn(0x7f0f140cfff0 legacy=0x7f0f1415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f140d0bb0 con 0x7f0f14143040 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f0efc0019f0 con 0x7f0f14143040 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 >> v1:172.21.15.123:6789/0 conn(0x7f0f14143040 legacy=0x7f0f1406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0ef8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ef8004800 0x7f0ef8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8005250 0x7f0ef800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f140d1200 con 0x7f0ef80039b0 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef800e3f0 con 0x7f0ef8005250 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.553+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef800e5f0 con 0x7f0ef8004800 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0ef8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f0a000640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ef8004800 0x7f0ef8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f13400640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8005250 0x7f0ef800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 -- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ef8004800 msgr2=0x7f0ef8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ef8004800 0x7f0ef8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 -- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8005250 msgr2=0x7f0ef800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8005250 0x7f0ef800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f12a00640 1 -- 172.21.15.123:0/1195566816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef8003520 con 0x7f0ef80039b0 2024-04-25T19:02:36.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f13400640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8005250 0x7f0ef800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:36.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.554+0000 7f0f13400640 1 -- 172.21.15.123:0/1195566816 reap_dead start 2024-04-25T19:02:36.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.555+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0ef8003e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f04001720 tx=0x7f0ef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc0949951b7480e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.555+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04001370 con 0x7f0ef80039b0 2024-04-25T19:02:36.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.555+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f040052d0 con 0x7f0ef80039b0 2024-04-25T19:02:36.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.555+0000 7f0f12000640 1 -- 172.21.15.123:0/1195566816 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0402ab90 con 0x7f0ef80039b0 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 -- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 msgr2=0x7f0ef8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0ef8003e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f04001720 tx=0x7f0ef8005830 comp rx=0 tx=0).stop 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 -- 172.21.15.123:0/1195566816 shutdown_connections 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1195566816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0ef8003e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 -- 172.21.15.123:0/1195566816 >> 172.21.15.123:0/1195566816 conn(0x7f0f14073090 msgr2=0x7f0f140734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 -- 172.21.15.123:0/1195566816 shutdown_connections 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.556+0000 7f0f13e00640 1 -- 172.21.15.123:0/1195566816 wait complete. 2024-04-25T19:02:36.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.558+0000 7f0f13e00640 1 Processor -- start 2024-04-25T19:02:36.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.558+0000 7f0f13e00640 1 -- start start 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f140c6460 0x7f0f141fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 0x7f0f140c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14143450 con 0x7f0ef80039b0 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1415dda0 con 0x7f0f140c6460 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f13e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f140ca400 con 0x7f0f141fe2f0 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39990/0 (socket says 172.21.15.123:39990) 2024-04-25T19:02:36.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.560+0000 7f0f12a00640 1 -- 172.21.15.123:0/1451136468 learned_addr learned my addr 172.21.15.123:0/1451136468 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f0a000640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f140c6460 0x7f0f141fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f13400640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 0x7f0f140c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 -- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 msgr2=0x7f0f140c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 0x7f0f140c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 -- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f140c6460 msgr2=0x7f0f141fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f140c6460 0x7f0f141fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 -- 172.21.15.123:0/1451136468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141fe700 con 0x7f0ef80039b0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f13400640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 0x7f0f140c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f12a00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005040 tx=0x7f0f04001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04037040 con 0x7f0ef80039b0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f04040070 con 0x7f0ef80039b0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f14153b50 con 0x7f0ef80039b0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04007a30 con 0x7f0ef80039b0 2024-04-25T19:02:36.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.561+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f140c7170 con 0x7f0ef80039b0 2024-04-25T19:02:36.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.563+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14050740 con 0x7f0ef80039b0 2024-04-25T19:02:36.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.564+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f040052d0 con 0x7f0ef80039b0 2024-04-25T19:02:36.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.565+0000 7f0f0b400640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 0x7f0ef00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:36.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.565+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f0f040b4df0 con 0x7f0ef80039b0 2024-04-25T19:02:36.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.565+0000 7f0f0a000640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 0x7f0ef00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:36.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.565+0000 7f0f0a000640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 0x7f0ef00769c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0013d0 tx=0x7f0efc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:36.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.569+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f141fe700 con 0x7f0ef80039b0 2024-04-25T19:02:36.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.767+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f140b13b0 con 0x7f0ef80039b0 2024-04-25T19:02:36.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.768+0000 7f0f0b400640 1 -- 172.21.15.123:0/1451136468 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f0f0407f830 con 0x7f0ef80039b0 2024-04-25T19:02:36.770 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:02:36.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 msgr2=0x7f0ef00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 0x7f0ef00769c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0013d0 tx=0x7f0efc009040 comp rx=0 tx=0).stop 2024-04-25T19:02:36.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 msgr2=0x7f0f14153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:36.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005040 tx=0x7f0f04001120 comp rx=0 tx=0).stop 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 shutdown_connections 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ef00744b0 0x7f0ef00769c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f141fe2f0 0x7f0f140c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f140c6460 0x7f0f141fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 --2- 172.21.15.123:0/1451136468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef80039b0 0x7f0f14153610 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.770+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 >> 172.21.15.123:0/1451136468 conn(0x7f0f14073090 msgr2=0x7f0f140b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.771+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 shutdown_connections 2024-04-25T19:02:36.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:36.771+0000 7f0f13e00640 1 -- 172.21.15.123:0/1451136468 wait complete. 2024-04-25T19:02:36.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T19:02:37.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.052+0000 7f621a600640 1 Processor -- start 2024-04-25T19:02:37.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.052+0000 7f621a600640 1 -- start start 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f621a600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f621415d7a0 0x7f621406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f621a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621406d340 0x7f621406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f621a600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621406d750 con 0x7f6214156cd0 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f621a600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f621415dbb0 con 0x7f621415d7a0 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f621a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f62140afe00 con 0x7f621406d340 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f6213400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6214156cd0 0x7f6214070e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57794/0 (socket says 172.21.15.123:57794) 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f6213400640 1 -- 172.21.15.123:0/1131275393 learned_addr learned my addr 172.21.15.123:0/1131275393 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.054+0000 7f6212a00640 1 --2- 172.21.15.123:0/1131275393 >> v2:172.21.15.123:3300/0 conn(0x7f621415d7a0 0x7f621406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6213e00640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621406d340 0x7f621406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212000640 1 -- 172.21.15.123:0/1131275393 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469937178 0 0) 0x7f620c001110 con 0x7f6214156cd0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212000640 1 -- 172.21.15.123:0/1131275393 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61f4003520 con 0x7f6214156cd0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212a00640 1 -- 172.21.15.123:0/1131275393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621406d340 msgr2=0x7f621406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212a00640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621406d340 0x7f621406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212a00640 1 -- 172.21.15.123:0/1131275393 >> v1:172.21.15.123:6789/0 conn(0x7f6214156cd0 legacy=0x7f6214070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.055+0000 7f6212a00640 1 -- 172.21.15.123:0/1131275393 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62140b00a0 con 0x7f621415d7a0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.056+0000 7f6212a00640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621415d7a0 0x7f621406ce00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009d90 tx=0x7f61fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b51607929f901a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.056+0000 7f6212000640 1 -- 172.21.15.123:0/1131275393 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc007740 con 0x7f621415d7a0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.056+0000 7f6212000640 1 -- 172.21.15.123:0/1131275393 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc0078e0 con 0x7f621415d7a0 2024-04-25T19:02:37.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.056+0000 7f6212000640 1 -- 172.21.15.123:0/1131275393 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc007cb0 con 0x7f621415d7a0 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 -- 172.21.15.123:0/1131275393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621415d7a0 msgr2=0x7f621406ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621415d7a0 0x7f621406ce00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009d90 tx=0x7f61fc02a660 comp rx=0 tx=0).stop 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 -- 172.21.15.123:0/1131275393 shutdown_connections 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621406d340 0x7f621406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 --2- 172.21.15.123:0/1131275393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621415d7a0 0x7f621406ce00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 -- 172.21.15.123:0/1131275393 >> 172.21.15.123:0/1131275393 conn(0x7f62140d1ae0 msgr2=0x7f62140d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 -- 172.21.15.123:0/1131275393 shutdown_connections 2024-04-25T19:02:37.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.057+0000 7f621a600640 1 -- 172.21.15.123:0/1131275393 wait complete. 2024-04-25T19:02:37.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.059+0000 7f621a600640 1 Processor -- start 2024-04-25T19:02:37.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.060+0000 7f621a600640 1 -- start start 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6214156cd0 0x7f621414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415d7a0 0x7f621414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62140b0940 con 0x7f621406d340 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f62140b0660 con 0x7f621415d7a0 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f621a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f62140b0e60 con 0x7f6214156cd0 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f6213400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f6213400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40010/0 (socket says 172.21.15.123:40010) 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.061+0000 7f6213400640 1 -- 172.21.15.123:0/516691714 learned_addr learned my addr 172.21.15.123:0/516691714 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6212a00640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6214156cd0 0x7f621414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 -- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6214156cd0 msgr2=0x7f621414f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6214156cd0 0x7f621414f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 -- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415d7a0 msgr2=0x7f621414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415d7a0 0x7f621414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 -- 172.21.15.123:0/516691714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6214152150 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f6213400640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f620c00c3c0 tx=0x7f620c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c017070 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c005ce0 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0072f0 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6214152670 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.062+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6214152c30 con 0x7f621406d340 2024-04-25T19:02:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.063+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6214076a40 con 0x7f621406d340 2024-04-25T19:02:37.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.065+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f620c01a070 con 0x7f621406d340 2024-04-25T19:02:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.066+0000 7f620b400640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 0x7f61f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.066+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f620c094380 con 0x7f621406d340 2024-04-25T19:02:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.066+0000 7f6212a00640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 0x7f61f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.067+0000 7f6212a00640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 0x7f61f00769c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61fc034000 tx=0x7f61fc033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.071+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f621406d750 con 0x7f621406d340 2024-04-25T19:02:37.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.271+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f62140b61d0 con 0x7f621406d340 2024-04-25T19:02:37.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.271+0000 7f620b400640 1 -- 172.21.15.123:0/516691714 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f620c05edc0 con 0x7f621406d340 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:02:37.274 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-25T19:02:37.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.273+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 msgr2=0x7f61f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.273+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 0x7f61f00769c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61fc034000 tx=0x7f61fc033040 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 msgr2=0x7f6214151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f620c00c3c0 tx=0x7f620c00c880 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 shutdown_connections 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61f00744b0 0x7f61f00769c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415d7a0 0x7f621414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6214156cd0 0x7f621414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 --2- 172.21.15.123:0/516691714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621406d340 0x7f6214151c10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 >> 172.21.15.123:0/516691714 conn(0x7f62140d1ae0 msgr2=0x7f62140b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 shutdown_connections 2024-04-25T19:02:37.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.274+0000 7f621a600640 1 -- 172.21.15.123:0/516691714 wait complete. 2024-04-25T19:02:37.300 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T19:02:37.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T19:02:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.569+0000 7f71fd800640 1 Processor -- start 2024-04-25T19:02:37.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.569+0000 7f71fd800640 1 -- start start 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71fd800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71f8070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71fd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f8163fc0 con 0x7f71f806d580 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71fd800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f81642a0 con 0x7f71f8160820 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71fd800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71f80cf2a0 con 0x7f71f8156f00 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71f7400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f71f806d580 0x7f71f806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57800/0 (socket says 172.21.15.123:57800) 2024-04-25T19:02:37.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71f7400640 1 -- 172.21.15.123:0/1054803833 learned_addr learned my addr 172.21.15.123:0/1054803833 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71f8070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.572+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439153151 0 0) 0x7f71f8163fc0 con 0x7f71f806d580 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71dc003520 con 0x7f71f806d580 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4204402578 0 0) 0x7f71dc003520 con 0x7f71f806d580 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 msgr2=0x7f71f8070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71f8070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 >> v1:172.21.15.146:6789/0 conn(0x7f71f8160820 legacy=0x7f71f8160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f8164580 con 0x7f71f806d580 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f71e40019f0 con 0x7f71f806d580 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 >> v1:172.21.15.123:6789/0 conn(0x7f71f806d580 legacy=0x7f71f806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71dc0039e0 0x7f71dc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc004800 0x7f71dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71dc0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71f80cf2a0 con 0x7f71dc0039e0 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc00bb80 con 0x7f71dc004800 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc00bd80 con 0x7f71f8156f00 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.573+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71dc0039e0 0x7f71dc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71fce00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71dc0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7400640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc004800 0x7f71dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7e00640 1 -- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 msgr2=0x7f71dc0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f8156f00 0x7f71dc0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7e00640 1 -- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc004800 msgr2=0x7f71dc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc004800 0x7f71dc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7e00640 1 -- 172.21.15.123:0/1054803833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc003520 con 0x7f71dc0039e0 2024-04-25T19:02:37.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.574+0000 7f71f7400640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc004800 0x7f71dc004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.575+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71dc0039e0 0x7f71dc003e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009ac0 tx=0x7f71dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510f85b6dfeaa5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.575+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8036070 con 0x7f71dc0039e0 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.575+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e8031400 con 0x7f71dc0039e0 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.575+0000 7f71f6a00640 1 -- 172.21.15.123:0/1054803833 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8009690 con 0x7f71dc0039e0 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 -- 172.21.15.123:0/1054803833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71dc0039e0 msgr2=0x7f71dc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 --2- 172.21.15.123:0/1054803833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71dc0039e0 0x7f71dc003e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009ac0 tx=0x7f71dc005830 comp rx=0 tx=0).stop 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fce00640 1 -- 172.21.15.123:0/1054803833 reap_dead start 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 -- 172.21.15.123:0/1054803833 shutdown_connections 2024-04-25T19:02:37.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 -- 172.21.15.123:0/1054803833 >> 172.21.15.123:0/1054803833 conn(0x7f71f8072fd0 msgr2=0x7f71f80733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 -- 172.21.15.123:0/1054803833 shutdown_connections 2024-04-25T19:02:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.576+0000 7f71fd800640 1 -- 172.21.15.123:0/1054803833 wait complete. 2024-04-25T19:02:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.578+0000 7f71fd800640 1 Processor -- start 2024-04-25T19:02:37.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.578+0000 7f71fd800640 1 -- start start 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc0039e0 0x7f71f80b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 0x7f71f815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8157490 con 0x7f71f814d9c0 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8157310 con 0x7f71dc0039e0 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71fd800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71f8161340 con 0x7f71f80d0720 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40028/0 (socket says 172.21.15.123:40028) 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.580+0000 7f71f7400640 1 -- 172.21.15.123:0/1664004502 learned_addr learned my addr 172.21.15.123:0/1664004502 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:37.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc0039e0 0x7f71f80b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71fce00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 0x7f71f815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 -- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 msgr2=0x7f71f815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 0x7f71f815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 -- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc0039e0 msgr2=0x7f71f80b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc0039e0 0x7f71f80b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 -- 172.21.15.123:0/1664004502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f80b4820 con 0x7f71f814d9c0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71fce00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 0x7f71f815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71f7400640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71e400b180 tx=0x7f71e400b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4019070 con 0x7f71f814d9c0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e40070d0 con 0x7f71f814d9c0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f80b4b00 con 0x7f71f814d9c0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4007480 con 0x7f71f814d9c0 2024-04-25T19:02:37.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.581+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f80b50c0 con 0x7f71f814d9c0 2024-04-25T19:02:37.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.582+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f8050740 con 0x7f71f814d9c0 2024-04-25T19:02:37.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.584+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71e401c070 con 0x7f71f814d9c0 2024-04-25T19:02:37.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.584+0000 7f71efe00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 0x7f71d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.585+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f71e4095320 con 0x7f71f814d9c0 2024-04-25T19:02:37.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.585+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 0x7f71d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.585+0000 7f71f7e00640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 0x7f71d80769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f71e8033000 tx=0x7f71e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.589+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71e405fd60 con 0x7f71f814d9c0 2024-04-25T19:02:37.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.790+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f71f80b2090 con 0x7f71f814d9c0 2024-04-25T19:02:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:37.998+0000 7f71efe00640 1 -- 172.21.15.123:0/1664004502 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f71e405f660 con 0x7f71f814d9c0 2024-04-25T19:02:38.001 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-25T19:02:38.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.002+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 msgr2=0x7f71d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.002+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 0x7f71d80769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f71e8033000 tx=0x7f71e8008030 comp rx=0 tx=0).stop 2024-04-25T19:02:38.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.002+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 msgr2=0x7f71f81fa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.002+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71e400b180 tx=0x7f71e400b640 comp rx=0 tx=0).stop 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 shutdown_connections 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f71d80744b0 0x7f71d80769c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71f80d0720 0x7f71f815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f814d9c0 0x7f71f81fa430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 --2- 172.21.15.123:0/1664004502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71dc0039e0 0x7f71f80b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 >> 172.21.15.123:0/1664004502 conn(0x7f71f8072fd0 msgr2=0x7f71f80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 shutdown_connections 2024-04-25T19:02:38.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.003+0000 7f71fd800640 1 -- 172.21.15.123:0/1664004502 wait complete. 2024-04-25T19:02:38.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T19:02:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.302+0000 7ff7c2a00640 1 Processor -- start 2024-04-25T19:02:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.302+0000 7ff7c2a00640 1 -- start start 2024-04-25T19:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7bc146f10 0x7ff7bc162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7bc1618e0 con 0x7ff7bc06dc90 2024-04-25T19:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7bc0d1240 con 0x7ff7bc144310 2024-04-25T19:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff7bc070790 con 0x7ff7bc146f10 2024-04-25T19:02:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7bc146f10 0x7ff7bc162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7bc146f10 0x7ff7bc162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40036/0 (socket says 172.21.15.123:40036) 2024-04-25T19:02:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.304+0000 7ff7c2000640 1 -- 172.21.15.123:0/2361090627 learned_addr learned my addr 172.21.15.123:0/2361090627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.305+0000 7ff7c2000640 1 -- 172.21.15.123:0/2361090627 >> v1:172.21.15.146:6789/0 conn(0x7ff7bc144310 legacy=0x7ff7bc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.305+0000 7ff7c2000640 1 -- 172.21.15.123:0/2361090627 >> v1:172.21.15.123:6789/0 conn(0x7ff7bc06dc90 legacy=0x7ff7bc144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.305+0000 7ff7c2000640 1 -- 172.21.15.123:0/2361090627 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc161b50 con 0x7ff7bc146f10 2024-04-25T19:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.306+0000 7ff7c2000640 1 --2- 172.21.15.123:0/2361090627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc146f10 0x7ff7bc162350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140720096f747571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.306+0000 7ff7bbe00640 1 -- 172.21.15.123:0/2361090627 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4036070 con 0x7ff7bc146f10 2024-04-25T19:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.306+0000 7ff7bbe00640 1 -- 172.21.15.123:0/2361090627 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b4031600 con 0x7ff7bc146f10 2024-04-25T19:02:38.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.306+0000 7ff7bbe00640 1 -- 172.21.15.123:0/2361090627 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40319d0 con 0x7ff7bc146f10 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 -- 172.21.15.123:0/2361090627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc146f10 msgr2=0x7ff7bc162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/2361090627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc146f10 0x7ff7bc162350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b402a760 comp rx=0 tx=0).stop 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 -- 172.21.15.123:0/2361090627 shutdown_connections 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/2361090627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc146f10 0x7ff7bc162350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 -- 172.21.15.123:0/2361090627 >> 172.21.15.123:0/2361090627 conn(0x7ff7bc0730d0 msgr2=0x7ff7bc0734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 -- 172.21.15.123:0/2361090627 shutdown_connections 2024-04-25T19:02:38.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.307+0000 7ff7c2a00640 1 -- 172.21.15.123:0/2361090627 wait complete. 2024-04-25T19:02:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.310+0000 7ff7c2a00640 1 Processor -- start 2024-04-25T19:02:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.310+0000 7ff7c2a00640 1 -- start start 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7bc146f10 0x7ff7bc1e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7bc1e7a50 0x7ff7bc1e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc147460 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc070a70 con 0x7ff7bc146f10 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.311+0000 7ff7c2a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc1475e0 con 0x7ff7bc1e7a50 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40044/0 (socket says 172.21.15.123:40044) 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 -- 172.21.15.123:0/1926691880 learned_addr learned my addr 172.21.15.123:0/1926691880 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c1600640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7bc146f10 0x7ff7bc1e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c0c00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7bc1e7a50 0x7ff7bc1e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 -- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7bc1e7a50 msgr2=0x7ff7bc1e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7bc1e7a50 0x7ff7bc1e7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 -- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7bc146f10 msgr2=0x7ff7bc1e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7bc146f10 0x7ff7bc1e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 -- 172.21.15.123:0/1926691880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc1605a0 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.312+0000 7ff7c2000640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009f70 tx=0x7ff7b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.313+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4036050 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.313+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b4042070 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.313+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4009510 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.313+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc160ab0 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.313+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc161070 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.314+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc160820 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.315+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff7b4007bd0 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.316+0000 7ff7b9600640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 0x7ff79c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.316+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7ff7b40b5f70 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.317+0000 7ff7c1600640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 0x7ff79c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:38.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.317+0000 7ff7c1600640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 0x7ff79c0769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006fd0 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:38.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.322+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7b40808e0 con 0x7ff7bc1e83a0 2024-04-25T19:02:38.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:38.521+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff7bc050740 con 0x7ff7bc1e83a0 2024-04-25T19:02:39.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.004+0000 7ff7b9600640 1 -- 172.21.15.123:0/1926691880 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7ff7b40801e0 con 0x7ff7bc1e83a0 2024-04-25T19:02:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-25T19:02:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 msgr2=0x7ff79c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 0x7ff79c0769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006fd0 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-04-25T19:02:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 msgr2=0x7ff7bc13c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009f70 tx=0x7ff7b4004a40 comp rx=0 tx=0).stop 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 shutdown_connections 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff79c0744b0 0x7ff79c0769c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc1e83a0 0x7ff7bc13c190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7bc1e7a50 0x7ff7bc1e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 --2- 172.21.15.123:0/1926691880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7bc146f10 0x7ff7bc1e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.008+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 >> 172.21.15.123:0/1926691880 conn(0x7ff7bc0730d0 msgr2=0x7ff7bc06d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.009+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 shutdown_connections 2024-04-25T19:02:39.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.009+0000 7ff7c2a00640 1 -- 172.21.15.123:0/1926691880 wait complete. 2024-04-25T19:02:39.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T19:02:39.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.298+0000 7f9921400640 1 Processor -- start 2024-04-25T19:02:39.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.299+0000 7f9921400640 1 -- start start 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9921400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f991c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9921400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f991c0cd140 0x7f991c0af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9921400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c0cc5f0 con 0x7f991c162540 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9921400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f991c06deb0 con 0x7f991c0cd140 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9921400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f991c06d610 con 0x7f991c06daa0 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9919e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f991c162540 0x7f991c0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57816/0 (socket says 172.21.15.123:57816) 2024-04-25T19:02:39.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.301+0000 7f9919e00640 1 -- 172.21.15.123:0/1303192169 learned_addr learned my addr 172.21.15.123:0/1303192169 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:39.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f991a800640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f991c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f991b200640 1 --2- 172.21.15.123:0/1303192169 >> v2:172.21.15.146:3300/0 conn(0x7f991c0cd140 0x7f991c0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176264464 0 0) 0x7f991c0cc5f0 con 0x7f991c162540 2024-04-25T19:02:39.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9900003520 con 0x7f991c162540 2024-04-25T19:02:39.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1314724206 0 0) 0x7f9900003520 con 0x7f991c162540 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 msgr2=0x7f991c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f991c15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 >> v2:172.21.15.146:3300/0 conn(0x7f991c0cd140 msgr2=0x7f991c0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 --2- 172.21.15.123:0/1303192169 >> v2:172.21.15.146:3300/0 conn(0x7f991c0cd140 0x7f991c0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c06ca00 con 0x7f991c162540 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f9908001670 con 0x7f991c162540 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 >> v1:172.21.15.123:6789/0 conn(0x7f991c162540 legacy=0x7f991c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.302+0000 7f9919400640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99000039b0 0x7f9900003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919400640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9900004800 0x7f9900004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919400640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f9900005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06deb0 con 0x7f99000039b0 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06d610 con 0x7f9900004800 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f990000b9f0 con 0x7f991c06daa0 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99000039b0 0x7f9900003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f9919e00640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9900004800 0x7f9900004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991b200640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f9900005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 -- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 msgr2=0x7f9900005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c06daa0 0x7f9900005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 -- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9900004800 msgr2=0x7f9900004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9900004800 0x7f9900004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.303+0000 7f991a800640 1 -- 172.21.15.123:0/1303192169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9900003520 con 0x7f99000039b0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.304+0000 7f991a800640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99000039b0 0x7f9900003e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f990c009a90 tx=0x7f9900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865c5c2219f12ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c036070 con 0x7f99000039b0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c031400 con 0x7f99000039b0 2024-04-25T19:02:39.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9919400640 1 -- 172.21.15.123:0/1303192169 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c009690 con 0x7f99000039b0 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 -- 172.21.15.123:0/1303192169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99000039b0 msgr2=0x7f9900003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 --2- 172.21.15.123:0/1303192169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99000039b0 0x7f9900003e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f990c009a90 tx=0x7f9900005830 comp rx=0 tx=0).stop 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f991b200640 1 -- 172.21.15.123:0/1303192169 reap_dead start 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 -- 172.21.15.123:0/1303192169 shutdown_connections 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 -- 172.21.15.123:0/1303192169 >> 172.21.15.123:0/1303192169 conn(0x7f991c0ce390 msgr2=0x7f991c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 -- 172.21.15.123:0/1303192169 shutdown_connections 2024-04-25T19:02:39.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.305+0000 7f9921400640 1 -- 172.21.15.123:0/1303192169 wait complete. 2024-04-25T19:02:39.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.308+0000 7f9921400640 1 Processor -- start 2024-04-25T19:02:39.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.308+0000 7f9921400640 1 -- start start 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.309+0000 7f9921400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99000039b0 0x7f991c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9921400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9921400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c09dd00 0x7f991c09e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9921400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06d2a0 con 0x7f991c09b7d0 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9921400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06cfc0 con 0x7f99000039b0 2024-04-25T19:02:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9921400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f991c162950 con 0x7f991c09dd00 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40060/0 (socket says 172.21.15.123:40060) 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 -- 172.21.15.123:0/870180476 learned_addr learned my addr 172.21.15.123:0/870180476 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f991a800640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99000039b0 0x7f991c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f991b200640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c09dd00 0x7f991c09e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 -- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c09dd00 msgr2=0x7f991c09e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c09dd00 0x7f991c09e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 -- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99000039b0 msgr2=0x7f991c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99000039b0 0x7f991c14e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.310+0000 7f9919e00640 1 -- 172.21.15.123:0/870180476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c09c550 con 0x7f991c09b7d0 2024-04-25T19:02:39.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9919e00640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f990800e760 tx=0x7f990800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:39.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800c6c0 con 0x7f991c09b7d0 2024-04-25T19:02:39.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9908018070 con 0x7f991c09b7d0 2024-04-25T19:02:39.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c09c830 con 0x7f991c09b7d0 2024-04-25T19:02:39.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800caa0 con 0x7f991c09b7d0 2024-04-25T19:02:39.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.311+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c09cdf0 con 0x7f991c09b7d0 2024-04-25T19:02:39.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.312+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c09c2c0 con 0x7f991c09b7d0 2024-04-25T19:02:39.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.314+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f990800cc40 con 0x7f991c09b7d0 2024-04-25T19:02:39.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.315+0000 7f9912a00640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 0x7f98fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:39.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.315+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f9908094ff0 con 0x7f991c09b7d0 2024-04-25T19:02:39.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.315+0000 7f991a800640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 0x7f98fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:39.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.316+0000 7f991a800640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 0x7f98fc0769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f990c033000 tx=0x7f990c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:39.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.320+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f990805f760 con 0x7f991c09b7d0 2024-04-25T19:02:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:39.549+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f991c076a40 con 0x7f991c09b7d0 2024-04-25T19:02:40.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.050+0000 7f9912a00640 1 -- 172.21.15.123:0/870180476 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f990805f060 con 0x7f991c09b7d0 2024-04-25T19:02:40.057 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T19:02:40.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.057+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 msgr2=0x7f98fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.057+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 0x7f98fc0769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f990c033000 tx=0x7f990c008030 comp rx=0 tx=0).stop 2024-04-25T19:02:40.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.057+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 msgr2=0x7f991c09bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.057+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f990800e760 tx=0x7f990800ec20 comp rx=0 tx=0).stop 2024-04-25T19:02:40.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 shutdown_connections 2024-04-25T19:02:40.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f98fc0744b0 0x7f98fc0769c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c09dd00 0x7f991c09e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c09b7d0 0x7f991c09bbe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 --2- 172.21.15.123:0/870180476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99000039b0 0x7f991c14e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.059+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 >> 172.21.15.123:0/870180476 conn(0x7f991c0ce390 msgr2=0x7f991c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.060+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 shutdown_connections 2024-04-25T19:02:40.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.060+0000 7f9921400640 1 -- 172.21.15.123:0/870180476 wait complete. 2024-04-25T19:02:40.103 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T19:02:40.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T19:02:40.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.349+0000 7f022a400640 1 Processor -- start 2024-04-25T19:02:40.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.349+0000 7f022a400640 1 -- start start 2024-04-25T19:02:40.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.351+0000 7f022a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f022415e570 0x7f022415e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.351+0000 7f022a400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022415f460 con 0x7f0224154a20 2024-04-25T19:02:40.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.351+0000 7f022a400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022415eec0 con 0x7f02240b53a0 2024-04-25T19:02:40.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.351+0000 7f022a400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f022415f180 con 0x7f022415e570 2024-04-25T19:02:40.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f021be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f022415e570 0x7f022415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f0223e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0224154a20 0x7f0224154e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57832/0 (socket says 172.21.15.123:57832) 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f0223e00640 1 -- 172.21.15.123:0/2917854318 learned_addr learned my addr 172.21.15.123:0/2917854318 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f021be00640 1 -- 172.21.15.123:0/2917854318 >> v1:172.21.15.146:6789/0 conn(0x7f02240b53a0 legacy=0x7f022406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f021be00640 1 -- 172.21.15.123:0/2917854318 >> v1:172.21.15.123:6789/0 conn(0x7f0224154a20 legacy=0x7f0224154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f0222a00640 1 -- 172.21.15.123:0/2917854318 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277634781 0 0) 0x7f022415f460 con 0x7f0224154a20 2024-04-25T19:02:40.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.352+0000 7f021be00640 1 -- 172.21.15.123:0/2917854318 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02240b48c0 con 0x7f022415e570 2024-04-25T19:02:40.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.353+0000 7f021be00640 1 --2- 172.21.15.123:0/2917854318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f022415e570 0x7f022415e980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f020c0097a0 tx=0x7f020c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20db93b7dc3c187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.354+0000 7f0222a00640 1 -- 172.21.15.123:0/2917854318 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c037070 con 0x7f022415e570 2024-04-25T19:02:40.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.354+0000 7f0222a00640 1 -- 172.21.15.123:0/2917854318 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c032440 con 0x7f022415e570 2024-04-25T19:02:40.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.354+0000 7f0222a00640 1 -- 172.21.15.123:0/2917854318 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c02ae60 con 0x7f022415e570 2024-04-25T19:02:40.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.354+0000 7f022a400640 1 -- 172.21.15.123:0/2917854318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f022415e570 msgr2=0x7f022415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.354+0000 7f022a400640 1 --2- 172.21.15.123:0/2917854318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f022415e570 0x7f022415e980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f020c0097a0 tx=0x7f020c02a110 comp rx=0 tx=0).stop 2024-04-25T19:02:40.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.355+0000 7f022a400640 1 -- 172.21.15.123:0/2917854318 shutdown_connections 2024-04-25T19:02:40.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.355+0000 7f022a400640 1 --2- 172.21.15.123:0/2917854318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f022415e570 0x7f022415e980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.355+0000 7f022a400640 1 -- 172.21.15.123:0/2917854318 >> 172.21.15.123:0/2917854318 conn(0x7f02240d1f80 msgr2=0x7f02240732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.355+0000 7f022a400640 1 -- 172.21.15.123:0/2917854318 shutdown_connections 2024-04-25T19:02:40.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.355+0000 7f022a400640 1 -- 172.21.15.123:0/2917854318 wait complete. 2024-04-25T19:02:40.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.357+0000 7f022a400640 1 Processor -- start 2024-04-25T19:02:40.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.358+0000 7f022a400640 1 -- start start 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.359+0000 7f022a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.359+0000 7f022a400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0224154a20 0x7f02241fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.359+0000 7f022a400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 0x7f02241f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f022a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f022406de70 con 0x7f02240b53a0 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f022a400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02240b4e80 con 0x7f022415e570 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f022a400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02240cd770 con 0x7f0224154a20 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40076/0 (socket says 172.21.15.123:40076) 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 -- 172.21.15.123:0/3204226915 learned_addr learned my addr 172.21.15.123:0/3204226915 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f021be00640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0224154a20 0x7f02241fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 -- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0224154a20 msgr2=0x7f02241fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223e00640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 0x7f02241f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0224154a20 0x7f02241fa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 -- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 msgr2=0x7f02241f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 0x7f02241f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223400640 1 -- 172.21.15.123:0/3204226915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0224159320 con 0x7f02240b53a0 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.360+0000 7f0223e00640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 0x7f02241f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:40.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f0223400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f021400c430 tx=0x7f021400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214017070 con 0x7f02240b53a0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02140052d0 con 0x7f02240b53a0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214003830 con 0x7f02240b53a0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241597a0 con 0x7f02240b53a0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.361+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0224159040 con 0x7f02240b53a0 2024-04-25T19:02:40.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.362+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0224076a40 con 0x7f02240b53a0 2024-04-25T19:02:40.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.364+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f021401a070 con 0x7f02240b53a0 2024-04-25T19:02:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.365+0000 7f021b400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 0x7f02000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.365+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0214094490 con 0x7f02240b53a0 2024-04-25T19:02:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.366+0000 7f021be00640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 0x7f02000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.366+0000 7f021be00640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 0x7f02000769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f020c034000 tx=0x7f020c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.370+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f021405ec90 con 0x7f02240b53a0 2024-04-25T19:02:40.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:40.579+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f022406e350 con 0x7f02240b53a0 2024-04-25T19:02:41.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.054+0000 7f021b400640 1 -- 172.21.15.123:0/3204226915 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f021405e590 con 0x7f02240b53a0 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 msgr2=0x7f02000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 0x7f02000769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f020c034000 tx=0x7f020c02e000 comp rx=0 tx=0).stop 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 msgr2=0x7f022415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f021400c430 tx=0x7f021400c8f0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 shutdown_connections 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f02000744b0 0x7f02000769c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f022415e570 0x7f02241f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0224154a20 0x7f02241fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 --2- 172.21.15.123:0/3204226915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02240b53a0 0x7f022415b790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 >> 172.21.15.123:0/3204226915 conn(0x7f02240d1f80 msgr2=0x7f02240cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 shutdown_connections 2024-04-25T19:02:41.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.058+0000 7f022a400640 1 -- 172.21.15.123:0/3204226915 wait complete. 2024-04-25T19:02:41.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T19:02:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.362+0000 7f5a4b000640 1 Processor -- start 2024-04-25T19:02:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.363+0000 7f5a4b000640 1 -- start start 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a4b000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a4406daa0 0x7f5a4415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a4b000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 0x7f5a440af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a4b000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a440cc5f0 con 0x7f5a44162540 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a4b000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4406deb0 con 0x7f5a4406daa0 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a4b000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4406d610 con 0x7f5a440cd140 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a43e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a4406daa0 0x7f5a4415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a3b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5a44162540 0x7f5a440cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57844/0 (socket says 172.21.15.123:57844) 2024-04-25T19:02:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.365+0000 7f5a3b400640 1 -- 172.21.15.123:0/1451664331 learned_addr learned my addr 172.21.15.123:0/1451664331 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a49000640 1 --2- 172.21.15.123:0/1451664331 >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 0x7f5a440af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a43e00640 1 -- 172.21.15.123:0/1451664331 >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 msgr2=0x7f5a440af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a43e00640 1 --2- 172.21.15.123:0/1451664331 >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 0x7f5a440af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a43e00640 1 -- 172.21.15.123:0/1451664331 >> v1:172.21.15.123:6789/0 conn(0x7f5a44162540 legacy=0x7f5a440cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a43e00640 1 -- 172.21.15.123:0/1451664331 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4406ca00 con 0x7f5a4406daa0 2024-04-25T19:02:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.366+0000 7f5a49000640 1 --2- 172.21.15.123:0/1451664331 >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 0x7f5a440af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.367+0000 7f5a43e00640 1 --2- 172.21.15.123:0/1451664331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4406daa0 0x7f5a4415dfc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d01c0 tx=0x7f5a3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea94bfe9ed5f36a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.367+0000 7f5a43400640 1 -- 172.21.15.123:0/1451664331 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037070 con 0x7f5a4406daa0 2024-04-25T19:02:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.367+0000 7f5a43400640 1 -- 172.21.15.123:0/1451664331 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34032440 con 0x7f5a4406daa0 2024-04-25T19:02:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.367+0000 7f5a43400640 1 -- 172.21.15.123:0/1451664331 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34032810 con 0x7f5a4406daa0 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 -- 172.21.15.123:0/1451664331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4406daa0 msgr2=0x7f5a4415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 --2- 172.21.15.123:0/1451664331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4406daa0 0x7f5a4415dfc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d01c0 tx=0x7f5a3402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 -- 172.21.15.123:0/1451664331 shutdown_connections 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 --2- 172.21.15.123:0/1451664331 >> v2:172.21.15.146:3300/0 conn(0x7f5a440cd140 0x7f5a440af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 --2- 172.21.15.123:0/1451664331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a4406daa0 0x7f5a4415dfc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 -- 172.21.15.123:0/1451664331 >> 172.21.15.123:0/1451664331 conn(0x7f5a440ce390 msgr2=0x7f5a440ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 -- 172.21.15.123:0/1451664331 shutdown_connections 2024-04-25T19:02:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.368+0000 7f5a4b000640 1 -- 172.21.15.123:0/1451664331 wait complete. 2024-04-25T19:02:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.370+0000 7f5a4b000640 1 Processor -- start 2024-04-25T19:02:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.370+0000 7f5a4b000640 1 -- start start 2024-04-25T19:02:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4406daa0 0x7f5a44154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a44162540 0x7f5a44154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4406d2a0 con 0x7f5a440b61f0 2024-04-25T19:02:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4406cfc0 con 0x7f5a44162540 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a4b000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a44162950 con 0x7f5a4406daa0 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40098/0 (socket says 172.21.15.123:40098) 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 -- 172.21.15.123:0/2647060967 learned_addr learned my addr 172.21.15.123:0/2647060967 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 -- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4406daa0 msgr2=0x7f5a44154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a43e00640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4406daa0 0x7f5a44154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a3b400640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a44162540 0x7f5a44154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4406daa0 0x7f5a44154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 -- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a44162540 msgr2=0x7f5a44154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a44162540 0x7f5a44154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.372+0000 7f5a49000640 1 -- 172.21.15.123:0/2647060967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a44156f70 con 0x7f5a440b61f0 2024-04-25T19:02:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a49000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00cd80 tx=0x7f5a3c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00f680 con 0x7f5a440b61f0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3c00f840 con 0x7f5a440b61f0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00fc30 con 0x7f5a440b61f0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a44157250 con 0x7f5a440b61f0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.373+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a44155190 con 0x7f5a440b61f0 2024-04-25T19:02:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.375+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a44156ce0 con 0x7f5a440b61f0 2024-04-25T19:02:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.376+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a3c0213f0 con 0x7f5a440b61f0 2024-04-25T19:02:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.377+0000 7f5a40c00640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 0x7f5a20076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.377+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f5a3c095fa0 con 0x7f5a440b61f0 2024-04-25T19:02:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.377+0000 7f5a43e00640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 0x7f5a20076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.378+0000 7f5a43e00640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 0x7f5a20076a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a3402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.382+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a3c014030 con 0x7f5a440b61f0 2024-04-25T19:02:41.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:41.580+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5a44069c60 con 0x7f5a440b61f0 2024-04-25T19:02:43.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.099+0000 7f5a40c00640 1 -- 172.21.15.123:0/2647060967 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f5a3c060770 con 0x7f5a440b61f0 2024-04-25T19:02:43.101 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' created 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 msgr2=0x7f5a20076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 0x7f5a20076a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a3402e000 comp rx=0 tx=0).stop 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 msgr2=0x7f5a44153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00cd80 tx=0x7f5a3c00a650 comp rx=0 tx=0).stop 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 shutdown_connections 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5a20074500 0x7f5a20076a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a440b61f0 0x7f5a44153530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a44162540 0x7f5a44154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 --2- 172.21.15.123:0/2647060967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4406daa0 0x7f5a44154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 >> 172.21.15.123:0/2647060967 conn(0x7f5a440ce390 msgr2=0x7f5a440d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.103+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 shutdown_connections 2024-04-25T19:02:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.104+0000 7f5a4b000640 1 -- 172.21.15.123:0/2647060967 wait complete. 2024-04-25T19:02:43.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T19:02:43.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.387+0000 7f752b200640 1 Processor -- start 2024-04-25T19:02:43.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.388+0000 7f752b200640 1 -- start start 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f752b200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f752c0b5550 0x7f752c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f752b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c160520 con 0x7f752c160110 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f752b200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c164ae0 con 0x7f752c0d13f0 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f752b200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f752c163f60 con 0x7f752c0b5550 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f7529e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f752c160110 0x7f752c0ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57856/0 (socket says 172.21.15.123:57856) 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.390+0000 7f7529e00640 1 -- 172.21.15.123:0/1784796276 learned_addr learned my addr 172.21.15.123:0/1784796276 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297724052 0 0) 0x7f752c160520 con 0x7f752c160110 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7510003520 con 0x7f752c160110 2024-04-25T19:02:43.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7523e00640 1 --2- 172.21.15.123:0/1784796276 >> v2:172.21.15.146:3300/0 conn(0x7f752c0b5550 0x7f752c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 463671805 0 0) 0x7f7510003520 con 0x7f752c160110 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 >> v2:172.21.15.146:3300/0 conn(0x7f752c0b5550 msgr2=0x7f752c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 --2- 172.21.15.123:0/1784796276 >> v2:172.21.15.146:3300/0 conn(0x7f752c0b5550 0x7f752c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 >> v1:172.21.15.146:6789/0 conn(0x7f752c0d13f0 legacy=0x7f752c0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c164240 con 0x7f752c160110 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f751c001670 con 0x7f752c160110 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 >> v1:172.21.15.123:6789/0 conn(0x7f752c160110 legacy=0x7f752c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75100039b0 0x7f7510003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510005350 0x7f751000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f752c163f60 con 0x7f7510004800 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f751000c280 con 0x7f7510005350 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f751000c480 con 0x7f75100039b0 2024-04-25T19:02:43.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.391+0000 7f7523e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f752a800640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510005350 0x7f751000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7529e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75100039b0 0x7f7510003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7523e00640 1 -- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75100039b0 msgr2=0x7f7510003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7523e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75100039b0 0x7f7510003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7523e00640 1 -- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510005350 msgr2=0x7f751000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7523e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510005350 0x7f751000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7523e00640 1 -- 172.21.15.123:0/1784796276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7510003520 con 0x7f7510004800 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f752a800640 1 -- 172.21.15.123:0/1784796276 reap_dead start 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.392+0000 7f7529e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75100039b0 0x7f7510003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.393+0000 7f7523e00640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 0x7f7510004c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f751400ba10 tx=0x7f7510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d9d39ff24568b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.393+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751400b5e0 con 0x7f7510004800 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.393+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751402adb0 con 0x7f7510004800 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.393+0000 7f7529400640 1 -- 172.21.15.123:0/1784796276 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514005bb0 con 0x7f7510004800 2024-04-25T19:02:43.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 -- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 msgr2=0x7f7510004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 0x7f7510004c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f751400ba10 tx=0x7f7510005830 comp rx=0 tx=0).stop 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 -- 172.21.15.123:0/1784796276 shutdown_connections 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 --2- 172.21.15.123:0/1784796276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 -- 172.21.15.123:0/1784796276 >> 172.21.15.123:0/1784796276 conn(0x7f752c0d02a0 msgr2=0x7f752c0d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 -- 172.21.15.123:0/1784796276 shutdown_connections 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.394+0000 7f752b200640 1 -- 172.21.15.123:0/1784796276 wait complete. 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.397+0000 7f752b200640 1 Processor -- start 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.397+0000 7f752b200640 1 -- start start 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 0x7f752c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 0x7f752c1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f752c163bf0 con 0x7f752c0c9090 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f752c164800 con 0x7f7510004800 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f752b200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f752c0b5960 con 0x7f752c06f060 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f7523e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f7523e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40124/0 (socket says 172.21.15.123:40124) 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.398+0000 7f7523e00640 1 -- 172.21.15.123:0/2105339562 learned_addr learned my addr 172.21.15.123:0/2105339562 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f752a800640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 0x7f752c1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7529e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 0x7f752c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 -- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 msgr2=0x7f752c1f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 0x7f752c1f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 -- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 msgr2=0x7f752c159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 0x7f752c159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 -- 172.21.15.123:0/2105339562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c06f470 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f752a800640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 0x7f752c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7529e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 0x7f752c159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7523e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f751402a8a0 tx=0x7f7514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514043070 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751402abb0 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514004480 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.399+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c0c7310 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.400+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c0c7780 con 0x7f752c0c9090 2024-04-25T19:02:43.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.401+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c0c74e0 con 0x7f752c0c9090 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.402+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7514046070 con 0x7f752c0c9090 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.403+0000 7f7522000640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 0x7f75080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.403+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f75140b5300 con 0x7f752c0c9090 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.403+0000 7f7529e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 0x7f75080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.404+0000 7f7529e00640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 0x7f75080769f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f751c0017e0 tx=0x7f751c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:43.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.408+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f752c0c74e0 con 0x7f752c0c9090 2024-04-25T19:02:43.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:43.608+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f752c050740 con 0x7f752c0c9090 2024-04-25T19:02:44.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.104+0000 7f7522000640 1 -- 172.21.15.123:0/2105339562 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f751404d080 con 0x7f752c0c9090 2024-04-25T19:02:44.106 INFO:teuthology.orchestra.run.smithi123.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T19:02:44.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.107+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 msgr2=0x7f75080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:44.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.107+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 0x7f75080769f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f751c0017e0 tx=0x7f751c009040 comp rx=0 tx=0).stop 2024-04-25T19:02:44.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.107+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 msgr2=0x7f752c06eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.107+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f751402a8a0 tx=0x7f7514004730 comp rx=0 tx=0).stop 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 shutdown_connections 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75080744e0 0x7f75080769f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f752c06f060 0x7f752c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f752c0c9090 0x7f752c06eb20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 --2- 172.21.15.123:0/2105339562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7510004800 0x7f752c159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 >> 172.21.15.123:0/2105339562 conn(0x7f752c0d02a0 msgr2=0x7f752c0cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:44.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 shutdown_connections 2024-04-25T19:02:44.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.108+0000 7f752b200640 1 -- 172.21.15.123:0/2105339562 wait complete. 2024-04-25T19:02:44.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T19:02:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.409+0000 7ffbb8a00640 1 Processor -- start 2024-04-25T19:02:44.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.409+0000 7ffbb8a00640 1 -- start start 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb8a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffbb415df10 0x7ffbb40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb8a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbb406d580 0x7ffbb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb8a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40d1430 con 0x7ffbb40b0700 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb8a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb4070f40 con 0x7ffbb406d580 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb8a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb4161e90 con 0x7ffbb415df10 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbb406d580 0x7ffbb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbb406d580 0x7ffbb406d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40132/0 (socket says 172.21.15.123:40132) 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 -- 172.21.15.123:0/4015057565 learned_addr learned my addr 172.21.15.123:0/4015057565 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:44.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb2800640 1 --2- 172.21.15.123:0/4015057565 >> v2:172.21.15.146:3300/0 conn(0x7ffbb415df10 0x7ffbb40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 -- 172.21.15.123:0/4015057565 >> v2:172.21.15.146:3300/0 conn(0x7ffbb415df10 msgr2=0x7ffbb40702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 --2- 172.21.15.123:0/4015057565 >> v2:172.21.15.146:3300/0 conn(0x7ffbb415df10 0x7ffbb40702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.412+0000 7ffbb3200640 1 -- 172.21.15.123:0/4015057565 >> v1:172.21.15.146:6789/0 conn(0x7ffbb40b0700 legacy=0x7ffbb4162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.413+0000 7ffbb3200640 1 -- 172.21.15.123:0/4015057565 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb406ded0 con 0x7ffbb406d580 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.414+0000 7ffbb3200640 1 --2- 172.21.15.123:0/4015057565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb406d580 0x7ffbb406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40517e0 tx=0x7ffbac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9681a969db66425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:44.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.414+0000 7ffbb1400640 1 -- 172.21.15.123:0/4015057565 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac036070 con 0x7ffbb406d580 2024-04-25T19:02:44.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.414+0000 7ffbb1400640 1 -- 172.21.15.123:0/4015057565 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac031600 con 0x7ffbb406d580 2024-04-25T19:02:44.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.414+0000 7ffbb1400640 1 -- 172.21.15.123:0/4015057565 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0319d0 con 0x7ffbb406d580 2024-04-25T19:02:44.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 -- 172.21.15.123:0/4015057565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb406d580 msgr2=0x7ffbb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/4015057565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb406d580 0x7ffbb406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40517e0 tx=0x7ffbac02a6f0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 -- 172.21.15.123:0/4015057565 shutdown_connections 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/4015057565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb406d580 0x7ffbb406d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/4015057565 >> v2:172.21.15.146:3300/0 conn(0x7ffbb415df10 0x7ffbb40702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 -- 172.21.15.123:0/4015057565 >> 172.21.15.123:0/4015057565 conn(0x7ffbb4073090 msgr2=0x7ffbb40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 -- 172.21.15.123:0/4015057565 shutdown_connections 2024-04-25T19:02:44.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.415+0000 7ffbb8a00640 1 -- 172.21.15.123:0/4015057565 wait complete. 2024-04-25T19:02:44.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.417+0000 7ffbb8a00640 1 Processor -- start 2024-04-25T19:02:44.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.417+0000 7ffbb8a00640 1 -- start start 2024-04-25T19:02:44.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbb406d580 0x7ffbb415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbb415df10 0x7ffbb41f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4161b20 con 0x7ffbb40b0700 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4070c60 con 0x7ffbb415df10 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb8a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40cdfe0 con 0x7ffbb406d580 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40142/0 (socket says 172.21.15.123:40142) 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.419+0000 7ffbb1e00640 1 -- 172.21.15.123:0/23642249 learned_addr learned my addr 172.21.15.123:0/23642249 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 -- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbb406d580 msgr2=0x7ffbb415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbb406d580 0x7ffbb415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 -- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbb415df10 msgr2=0x7ffbb41f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbb415df10 0x7ffbb41f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 -- 172.21.15.123:0/23642249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb415e860 con 0x7ffbb40b0700 2024-04-25T19:02:44.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb1e00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffba000b3e0 tx=0x7ffba000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0015020 con 0x7ffbb40b0700 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba0011850 con 0x7ffbb40b0700 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb41611d0 con 0x7ffbb40b0700 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0011c40 con 0x7ffbb40b0700 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.420+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb4160ef0 con 0x7ffbb40b0700 2024-04-25T19:02:44.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.421+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4050740 con 0x7ffbb40b0700 2024-04-25T19:02:44.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.423+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffba0018020 con 0x7ffbb40b0700 2024-04-25T19:02:44.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.423+0000 7ffbaaa00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:44.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.424+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7ffba0094f90 con 0x7ffbb40b0700 2024-04-25T19:02:44.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.424+0000 7ffbb2800640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:44.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.424+0000 7ffbb2800640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 0x7ffb940769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:44.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.429+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffba001f070 con 0x7ffbb40b0700 2024-04-25T19:02:44.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:44.670+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ffbb4074190 con 0x7ffbb40b0700 2024-04-25T19:02:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.135+0000 7ffbaaa00640 1 -- 172.21.15.123:0/23642249 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7ffba005f400 con 0x7ffbb40b0700 2024-04-25T19:02:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:added data pool 4 to fsmap 2024-04-25T19:02:45.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.138+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 msgr2=0x7ffb940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.138+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 0x7ffb940769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2024-04-25T19:02:45.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 msgr2=0x7ffbb41f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffba000b3e0 tx=0x7ffba000b8a0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 shutdown_connections 2024-04-25T19:02:45.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ffb940744e0 0x7ffb940769f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbb415df10 0x7ffbb41f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbb40b0700 0x7ffbb41f58c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 --2- 172.21.15.123:0/23642249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffbb406d580 0x7ffbb415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 >> 172.21.15.123:0/23642249 conn(0x7ffbb4073090 msgr2=0x7ffbb40b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 shutdown_connections 2024-04-25T19:02:45.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.139+0000 7ffbb8a00640 1 -- 172.21.15.123:0/23642249 wait complete. 2024-04-25T19:02:45.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:02:45.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.423+0000 7f0551600640 1 Processor -- start 2024-04-25T19:02:45.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.423+0000 7f0551600640 1 -- start start 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f0551600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f054c06fe20 0x7f054c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f0551600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 0x7f054c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f0551600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054c06ded0 con 0x7f054c15e2f0 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f0551600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f054c15ee10 con 0x7f054c06fe20 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f0551600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f054c15f070 con 0x7f054c06d580 2024-04-25T19:02:45.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f054c06fe20 0x7f054c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f054c06fe20 0x7f054c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39524/0 (socket says 172.21.15.123:39524) 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 -- 172.21.15.123:0/3460303972 learned_addr learned my addr 172.21.15.123:0/3460303972 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054b400640 1 --2- 172.21.15.123:0/3460303972 >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 0x7f054c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 -- 172.21.15.123:0/3460303972 >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 msgr2=0x7f054c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 --2- 172.21.15.123:0/3460303972 >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 0x7f054c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.426+0000 7f054be00640 1 -- 172.21.15.123:0/3460303972 >> v1:172.21.15.146:6789/0 conn(0x7f054c15e2f0 legacy=0x7f054c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.427+0000 7f054be00640 1 -- 172.21.15.123:0/3460303972 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c06f340 con 0x7f054c06fe20 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.427+0000 7f054b400640 1 --2- 172.21.15.123:0/3460303972 >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 0x7f054c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:45.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.428+0000 7f054be00640 1 --2- 172.21.15.123:0/3460303972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c06fe20 0x7f054c15b6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f054c070260 tx=0x7f053c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f323eb3bac563d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.428+0000 7f054aa00640 1 -- 172.21.15.123:0/3460303972 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c037070 con 0x7f054c06fe20 2024-04-25T19:02:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.428+0000 7f054aa00640 1 -- 172.21.15.123:0/3460303972 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053c032440 con 0x7f054c06fe20 2024-04-25T19:02:45.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f054aa00640 1 -- 172.21.15.123:0/3460303972 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c032810 con 0x7f054c06fe20 2024-04-25T19:02:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 -- 172.21.15.123:0/3460303972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c06fe20 msgr2=0x7f054c15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 --2- 172.21.15.123:0/3460303972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c06fe20 0x7f054c15b6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f054c070260 tx=0x7f053c02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 -- 172.21.15.123:0/3460303972 shutdown_connections 2024-04-25T19:02:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 --2- 172.21.15.123:0/3460303972 >> v2:172.21.15.146:3300/0 conn(0x7f054c06d580 0x7f054c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 --2- 172.21.15.123:0/3460303972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c06fe20 0x7f054c15b6b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.429+0000 7f0551600640 1 -- 172.21.15.123:0/3460303972 >> 172.21.15.123:0/3460303972 conn(0x7f054c073200 msgr2=0x7f054c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.430+0000 7f0551600640 1 -- 172.21.15.123:0/3460303972 shutdown_connections 2024-04-25T19:02:45.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.430+0000 7f0551600640 1 -- 172.21.15.123:0/3460303972 wait complete. 2024-04-25T19:02:45.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.432+0000 7f0551600640 1 Processor -- start 2024-04-25T19:02:45.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.432+0000 7f0551600640 1 -- start start 2024-04-25T19:02:45.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f054c06d580 0x7f054c139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f054c06fe20 0x7f054c13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f054c15bbf0 con 0x7f054c15e2f0 2024-04-25T19:02:45.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f054c06f900 con 0x7f054c06fe20 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0551600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f054c0ce500 con 0x7f054c06d580 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.434+0000 7f0550c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39536/0 (socket says 172.21.15.123:39536) 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 -- 172.21.15.123:0/3235523972 learned_addr learned my addr 172.21.15.123:0/3235523972 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f054b400640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f054c06fe20 0x7f054c13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f054be00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f054c06d580 0x7f054c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 -- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f054c06d580 msgr2=0x7f054c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f054c06d580 0x7f054c139f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 -- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f054c06fe20 msgr2=0x7f054c13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f054c06fe20 0x7f054c13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 -- 172.21.15.123:0/3235523972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c13d530 con 0x7f054c15e2f0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.435+0000 7f0550c00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f054400b9a0 tx=0x7f054400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.436+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544019070 con 0x7f054c15e2f0 2024-04-25T19:02:45.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.436+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054400cba0 con 0x7f054c15e2f0 2024-04-25T19:02:45.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.436+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544014440 con 0x7f054c15e2f0 2024-04-25T19:02:45.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.436+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c13dce0 con 0x7f054c15e2f0 2024-04-25T19:02:45.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.436+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1e78e0 con 0x7f054c15e2f0 2024-04-25T19:02:45.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.437+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c070a00 con 0x7f054c15e2f0 2024-04-25T19:02:45.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.438+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f054401c070 con 0x7f054c15e2f0 2024-04-25T19:02:45.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.439+0000 7f0543e00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 0x7f052c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.439+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f05440951c0 con 0x7f054c15e2f0 2024-04-25T19:02:45.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.439+0000 7f054be00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 0x7f052c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.440+0000 7f054be00640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 0x7f052c076a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f053c034000 tx=0x7f053c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.444+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f054405f6f0 con 0x7f054c15e2f0 2024-04-25T19:02:45.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.678+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f054c06d990 con 0x7f054c15e2f0 2024-04-25T19:02:45.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.679+0000 7f0543e00640 1 -- 172.21.15.123:0/3235523972 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f054405eff0 con 0x7f054c15e2f0 2024-04-25T19:02:45.681 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:45.681 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:02:44.672679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:02:45.682 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 msgr2=0x7f052c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 0x7f052c076a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f053c034000 tx=0x7f053c02e000 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 msgr2=0x7f054c1e83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f054400b9a0 tx=0x7f054400be60 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 shutdown_connections 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f052c074500 0x7f052c076a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f054c15e2f0 0x7f054c1e83c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f054c06fe20 0x7f054c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 --2- 172.21.15.123:0/3235523972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f054c06d580 0x7f054c139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 >> 172.21.15.123:0/3235523972 conn(0x7f054c073200 msgr2=0x7f054c160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 shutdown_connections 2024-04-25T19:02:45.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.682+0000 7f0551600640 1 -- 172.21.15.123:0/3235523972 wait complete. 2024-04-25T19:02:45.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:45.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.967+0000 7fec9ec00640 1 Processor -- start 2024-04-25T19:02:45.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.967+0000 7fec9ec00640 1 -- start start 2024-04-25T19:02:45.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ec00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec9806fe20 0x7fec9815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9815e2f0 0x7fec9815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ec00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9815f070 con 0x7fec9806d580 2024-04-25T19:02:45.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ec00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fec9806f340 con 0x7fec9806fe20 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ec00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec9806f620 con 0x7fec9815e2f0 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9d800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec9806fe20 0x7fec9815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.969+0000 7fec9ce00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fec9806d580 0x7fec9806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49402/0 (socket says 172.21.15.123:49402) 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9ce00640 1 -- 172.21.15.123:0/2705065161 learned_addr learned my addr 172.21.15.123:0/2705065161 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9e200640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9815e2f0 0x7fec9815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9d800640 1 -- 172.21.15.123:0/2705065161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9815e2f0 msgr2=0x7fec9815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9d800640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9815e2f0 0x7fec9815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9d800640 1 -- 172.21.15.123:0/2705065161 >> v1:172.21.15.123:6789/0 conn(0x7fec9806d580 legacy=0x7fec9806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec9d800640 1 -- 172.21.15.123:0/2705065161 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9806f900 con 0x7fec9806fe20 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.970+0000 7fec97e00640 1 -- 172.21.15.123:0/2705065161 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780259753 0 0) 0x7fec9815f070 con 0x7fec9806d580 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.971+0000 7fec9d800640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806fe20 0x7fec9815b6b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0097a0 tx=0x7fec8c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560dc5a5ec476d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.971+0000 7fec97e00640 1 -- 172.21.15.123:0/2705065161 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c037070 con 0x7fec9806fe20 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.971+0000 7fec97e00640 1 -- 172.21.15.123:0/2705065161 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c02ae50 con 0x7fec9806fe20 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.971+0000 7fec97e00640 1 -- 172.21.15.123:0/2705065161 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c032660 con 0x7fec9806fe20 2024-04-25T19:02:45.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 -- 172.21.15.123:0/2705065161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806fe20 msgr2=0x7fec9815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806fe20 0x7fec9815b6b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0097a0 tx=0x7fec8c02a100 comp rx=0 tx=0).stop 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 -- 172.21.15.123:0/2705065161 shutdown_connections 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9815e2f0 0x7fec9815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2705065161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806fe20 0x7fec9815b6b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 -- 172.21.15.123:0/2705065161 >> 172.21.15.123:0/2705065161 conn(0x7fec98073200 msgr2=0x7fec98073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 -- 172.21.15.123:0/2705065161 shutdown_connections 2024-04-25T19:02:45.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.972+0000 7fec9ec00640 1 -- 172.21.15.123:0/2705065161 wait complete. 2024-04-25T19:02:45.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.975+0000 7fec9ec00640 1 Processor -- start 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.975+0000 7fec9ec00640 1 -- start start 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9806fe20 0x7fec981368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 0x7fec980b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec9815edc0 con 0x7fec9806d580 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fec9815ec40 con 0x7fec9815e2f0 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9ec00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec980ce500 con 0x7fec9806fe20 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.976+0000 7fec9d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39546/0 (socket says 172.21.15.123:39546) 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 -- 172.21.15.123:0/2629176883 learned_addr learned my addr 172.21.15.123:0/2629176883 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9ce00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9806fe20 0x7fec981368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9e200640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 0x7fec980b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 -- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9806fe20 msgr2=0x7fec981368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9806fe20 0x7fec981368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 -- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 msgr2=0x7fec980b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 0x7fec980b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 -- 172.21.15.123:0/2629176883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec980cbf10 con 0x7fec9806d580 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9e200640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 0x7fec980b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.977+0000 7fec9d800640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec8c035040 tx=0x7fec8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.978+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c037040 con 0x7fec9806d580 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.978+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c004030 con 0x7fec9806d580 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.978+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec980cac10 con 0x7fec9806d580 2024-04-25T19:02:45.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.978+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c004420 con 0x7fec9806d580 2024-04-25T19:02:45.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.978+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec980cb1d0 con 0x7fec9806d580 2024-04-25T19:02:45.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.979+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec98070a00 con 0x7fec9806d580 2024-04-25T19:02:45.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.980+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec8c003610 con 0x7fec9806d580 2024-04-25T19:02:45.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.981+0000 7fec95600640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 0x7fec780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.981+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fec8c0b63c0 con 0x7fec9806d580 2024-04-25T19:02:45.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.981+0000 7fec9ce00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 0x7fec780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.982+0000 7fec9ce00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 0x7fec780769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec84000f60 tx=0x7fec84007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:45.986+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec8c080800 con 0x7fec9806d580 2024-04-25T19:02:46.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.190+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec980cf900 con 0x7fec9806d580 2024-04-25T19:02:46.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.193+0000 7fec95600640 1 -- 172.21.15.123:0/2629176883 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7fec8c080100 con 0x7fec9806d580 2024-04-25T19:02:46.196 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 msgr2=0x7fec780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 0x7fec780769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec84000f60 tx=0x7fec84007c00 comp rx=0 tx=0).stop 2024-04-25T19:02:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 msgr2=0x7fec980cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec8c035040 tx=0x7fec8c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 shutdown_connections 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fec780744b0 0x7fec780769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9815e2f0 0x7fec980b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9806fe20 0x7fec981368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 --2- 172.21.15.123:0/2629176883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9806d580 0x7fec980cb9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.196+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 >> 172.21.15.123:0/2629176883 conn(0x7fec98073200 msgr2=0x7fec9806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.197+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 shutdown_connections 2024-04-25T19:02:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.197+0000 7fec9ec00640 1 -- 172.21.15.123:0/2629176883 wait complete. 2024-04-25T19:02:46.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:46.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.478+0000 7f6a6ce00640 1 Processor -- start 2024-04-25T19:02:46.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.478+0000 7f6a6ce00640 1 -- start start 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a6ce00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a6ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a68154950 0x7f6a68154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a6ce00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a680aec30 con 0x7f6a68161b10 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a6ce00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6a680d1530 con 0x7f6a68154950 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a6ce00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a680aef10 con 0x7f6a680d1120 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a66800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a68154950 0x7f6a68154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a66800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a68154950 0x7f6a68154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39562/0 (socket says 172.21.15.123:39562) 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a66800640 1 -- 172.21.15.123:0/3777154152 learned_addr learned my addr 172.21.15.123:0/3777154152 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:46.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.481+0000 7f6a65e00640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.482+0000 7f6a66800640 1 -- 172.21.15.123:0/3777154152 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 msgr2=0x7f6a68070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:46.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.482+0000 7f6a66800640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.482+0000 7f6a66800640 1 -- 172.21.15.123:0/3777154152 >> v1:172.21.15.146:6789/0 conn(0x7f6a68161b10 legacy=0x7f6a68161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.482+0000 7f6a66800640 1 -- 172.21.15.123:0/3777154152 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a680af1f0 con 0x7f6a68154950 2024-04-25T19:02:46.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.483+0000 7f6a66800640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68154950 0x7f6a68154d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009e50 tx=0x7f6a6002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b40ff6c0380739a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:46.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.483+0000 7f6a65400640 1 -- 172.21.15.123:0/3777154152 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60036070 con 0x7f6a68154950 2024-04-25T19:02:46.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a65400640 1 -- 172.21.15.123:0/3777154152 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60031600 con 0x7f6a68154950 2024-04-25T19:02:46.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a65400640 1 -- 172.21.15.123:0/3777154152 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600319d0 con 0x7f6a68154950 2024-04-25T19:02:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3777154152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68154950 msgr2=0x7f6a68154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68154950 0x7f6a68154d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009e50 tx=0x7f6a6002a610 comp rx=0 tx=0).stop 2024-04-25T19:02:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3777154152 shutdown_connections 2024-04-25T19:02:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68154950 0x7f6a68154d60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3777154152 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3777154152 >> 172.21.15.123:0/3777154152 conn(0x7f6a680d1ae0 msgr2=0x7f6a680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:46.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3777154152 shutdown_connections 2024-04-25T19:02:46.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.484+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3777154152 wait complete. 2024-04-25T19:02:46.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.487+0000 7f6a6ce00640 1 Processor -- start 2024-04-25T19:02:46.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.487+0000 7f6a6ce00640 1 -- start start 2024-04-25T19:02:46.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a68154950 0x7f6a6815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681552a0 con 0x7f6a68161b10 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68071430 con 0x7f6a68154950 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.488+0000 7f6a6ce00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a68156c70 con 0x7f6a680d1120 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39574/0 (socket says 172.21.15.123:39574) 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 -- 172.21.15.123:0/3726853808 learned_addr learned my addr 172.21.15.123:0/3726853808 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a5d400640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a68154950 0x7f6a6815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a65e00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 -- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 msgr2=0x7f6a68160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 -- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a68154950 msgr2=0x7f6a6815b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a68154950 0x7f6a6815b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 -- 172.21.15.123:0/3726853808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a68156a60 con 0x7f6a68161b10 2024-04-25T19:02:46.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.489+0000 7f6a66800640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009f70 tx=0x7f6a60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.490+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6003f020 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.490+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60037040 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.490+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60036070 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.490+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a68154540 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.490+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a68160eb0 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.491+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a68160940 con 0x7f6a68161b10 2024-04-25T19:02:46.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.492+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a60042020 con 0x7f6a68161b10 2024-04-25T19:02:46.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.493+0000 7f6a5ea00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 0x7f6a440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.493+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6a600b5ae0 con 0x7f6a68161b10 2024-04-25T19:02:46.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.493+0000 7f6a65e00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 0x7f6a440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.494+0000 7f6a65e00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 0x7f6a440769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a58006fd0 tx=0x7f6a58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:46.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.497+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a60049030 con 0x7f6a68161b10 2024-04-25T19:02:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.696+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a680ae7e0 con 0x7f6a68161b10 2024-04-25T19:02:46.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.698+0000 7f6a5ea00640 1 -- 172.21.15.123:0/3726853808 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f6a680ae7e0 con 0x7f6a68161b10 2024-04-25T19:02:46.700 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:46.701 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:46.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 msgr2=0x7f6a440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 0x7f6a440769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a58006fd0 tx=0x7f6a58008040 comp rx=0 tx=0).stop 2024-04-25T19:02:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 msgr2=0x7f6a681500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009f70 tx=0x7f6a60004a40 comp rx=0 tx=0).stop 2024-04-25T19:02:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 shutdown_connections 2024-04-25T19:02:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a440744b0 0x7f6a440769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a68161b10 0x7f6a681500f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a68154950 0x7f6a6815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 --2- 172.21.15.123:0/3726853808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a680d1120 0x7f6a68160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 >> 172.21.15.123:0/3726853808 conn(0x7f6a680d1ae0 msgr2=0x7f6a680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.701+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 shutdown_connections 2024-04-25T19:02:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:46.702+0000 7f6a6ce00640 1 -- 172.21.15.123:0/3726853808 wait complete. 2024-04-25T19:02:46.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:02:47.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.015+0000 7fbef2200640 1 Processor -- start 2024-04-25T19:02:47.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.015+0000 7fbef2200640 1 -- start start 2024-04-25T19:02:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.017+0000 7fbef2200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbeec13fbf0 0x7fbeec06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.017+0000 7fbef2200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 0x7fbeec0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.017+0000 7fbef2200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeec06ded0 con 0x7fbeec06dac0 2024-04-25T19:02:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.017+0000 7fbef2200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec143050 con 0x7fbeec13fbf0 2024-04-25T19:02:47.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.017+0000 7fbef2200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0b5c70 con 0x7fbeec142c40 2024-04-25T19:02:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.018+0000 7fbef0e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbeec13fbf0 0x7fbeec06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.018+0000 7fbef1800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 0x7fbeec0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.018+0000 7fbef0e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbeec13fbf0 0x7fbeec06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:41524/0 (socket says 172.21.15.123:41524) 2024-04-25T19:02:47.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.018+0000 7fbef0e00640 1 -- 172.21.15.123:0/3393287596 learned_addr learned my addr 172.21.15.123:0/3393287596 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbef0e00640 1 -- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 msgr2=0x7fbeec0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbef0e00640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 0x7fbeec0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbef0e00640 1 -- 172.21.15.123:0/3393287596 >> v1:172.21.15.146:6789/0 conn(0x7fbeec06dac0 legacy=0x7fbeec142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbef0e00640 1 -- 172.21.15.123:0/3393287596 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0b5f50 con 0x7fbeec13fbf0 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbef1800640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 0x7fbeec0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:47.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.019+0000 7fbeeb400640 1 -- 172.21.15.123:0/3393287596 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700337667 0 0) 0x7fbed8001110 con 0x7fbeec06dac0 2024-04-25T19:02:47.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.020+0000 7fbef0e00640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec06d580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009870 tx=0x7fbee002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ae5a013075334a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.021+0000 7fbeeb400640 1 -- 172.21.15.123:0/3393287596 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0037070 con 0x7fbeec13fbf0 2024-04-25T19:02:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.021+0000 7fbeeb400640 1 -- 172.21.15.123:0/3393287596 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee0032440 con 0x7fbeec13fbf0 2024-04-25T19:02:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.021+0000 7fbeeb400640 1 -- 172.21.15.123:0/3393287596 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0032810 con 0x7fbeec13fbf0 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 -- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 msgr2=0x7fbeec06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec06d580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009870 tx=0x7fbee002a250 comp rx=0 tx=0).stop 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 -- 172.21.15.123:0/3393287596 shutdown_connections 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec142c40 0x7fbeec0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 --2- 172.21.15.123:0/3393287596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec06d580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 -- 172.21.15.123:0/3393287596 >> 172.21.15.123:0/3393287596 conn(0x7fbeec073200 msgr2=0x7fbeec073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 -- 172.21.15.123:0/3393287596 shutdown_connections 2024-04-25T19:02:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.022+0000 7fbef2200640 1 -- 172.21.15.123:0/3393287596 wait complete. 2024-04-25T19:02:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.025+0000 7fbef2200640 1 Processor -- start 2024-04-25T19:02:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.025+0000 7fbef2200640 1 -- start start 2024-04-25T19:02:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec06dac0 0x7fbeec13e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec13e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0b6a30 con 0x7fbeec142c40 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0b6510 con 0x7fbeec13fbf0 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.026+0000 7fbef2200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0b4b80 con 0x7fbeec06dac0 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39582/0 (socket says 172.21.15.123:39582) 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 -- 172.21.15.123:0/544534110 learned_addr learned my addr 172.21.15.123:0/544534110 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbeebe00640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec13e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef0e00640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec06dac0 0x7fbeec13e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 -- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec06dac0 msgr2=0x7fbeec13e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec06dac0 0x7fbeec13e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 -- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 msgr2=0x7fbeec13e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec13e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 -- 172.21.15.123:0/544534110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1f9d90 con 0x7fbeec142c40 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.027+0000 7fbef1800640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009e60 tx=0x7fbee400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.028+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4018030 con 0x7fbeec142c40 2024-04-25T19:02:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.028+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee400cdf0 con 0x7fbeec142c40 2024-04-25T19:02:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.028+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4003bb0 con 0x7fbeec142c40 2024-04-25T19:02:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.028+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1fa070 con 0x7fbeec142c40 2024-04-25T19:02:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.028+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1fa630 con 0x7fbeec142c40 2024-04-25T19:02:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.029+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec0b30f0 con 0x7fbeec142c40 2024-04-25T19:02:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.030+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbee401b070 con 0x7fbeec142c40 2024-04-25T19:02:47.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.031+0000 7fbee8c00640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 0x7fbecc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.032+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbee4095ff0 con 0x7fbeec142c40 2024-04-25T19:02:47.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.032+0000 7fbef0e00640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 0x7fbecc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.032+0000 7fbef0e00640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 0x7fbecc0769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbee0034000 tx=0x7fbee002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.037+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbeec0b30f0 con 0x7fbeec142c40 2024-04-25T19:02:47.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.276+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeec050740 con 0x7fbeec142c40 2024-04-25T19:02:47.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.277+0000 7fbee8c00640 1 -- 172.21.15.123:0/544534110 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7fbee4010040 con 0x7fbeec142c40 2024-04-25T19:02:47.279 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:47.279 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:02:44.672679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:02:47.279 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-04-25T19:02:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.279+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 msgr2=0x7fbecc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.279+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 0x7fbecc0769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbee0034000 tx=0x7fbee002e000 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.279+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 msgr2=0x7fbeec13ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009e60 tx=0x7fbee400c490 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 shutdown_connections 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbecc0744b0 0x7fbecc0769c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec142c40 0x7fbeec13ecf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeec13fbf0 0x7fbeec13e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 --2- 172.21.15.123:0/544534110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeec06dac0 0x7fbeec13e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 >> 172.21.15.123:0/544534110 conn(0x7fbeec073200 msgr2=0x7fbeec139580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 shutdown_connections 2024-04-25T19:02:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.280+0000 7fbef2200640 1 -- 172.21.15.123:0/544534110 wait complete. 2024-04-25T19:02:47.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:47.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.572+0000 7fc867e00640 1 Processor -- start 2024-04-25T19:02:47.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.572+0000 7fc867e00640 1 -- start start 2024-04-25T19:02:47.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.574+0000 7fc867e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc8600b57f0 0x7fc8600b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.574+0000 7fc867e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8600ae7e0 0x7fc8600aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.574+0000 7fc867e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8600b6140 con 0x7fc86015df00 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.574+0000 7fc867e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc86015e310 con 0x7fc8600ae7e0 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.574+0000 7fc867e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc8600b5360 con 0x7fc8600b57f0 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8600ae7e0 0x7fc8600aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8600ae7e0 0x7fc8600aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39596/0 (socket says 172.21.15.123:39596) 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 -- 172.21.15.123:0/3108174427 learned_addr learned my addr 172.21.15.123:0/3108174427 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc864a00640 1 --2- 172.21.15.123:0/3108174427 >> v2:172.21.15.146:3300/0 conn(0x7fc8600b57f0 0x7fc8600b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 -- 172.21.15.123:0/3108174427 >> v2:172.21.15.146:3300/0 conn(0x7fc8600b57f0 msgr2=0x7fc8600b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 --2- 172.21.15.123:0/3108174427 >> v2:172.21.15.146:3300/0 conn(0x7fc8600b57f0 0x7fc8600b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 -- 172.21.15.123:0/3108174427 >> v1:172.21.15.146:6789/0 conn(0x7fc86015df00 legacy=0x7fc86006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:47.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.575+0000 7fc865e00640 1 -- 172.21.15.123:0/3108174427 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600b4750 con 0x7fc8600ae7e0 2024-04-25T19:02:47.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.576+0000 7fc865e00640 1 --2- 172.21.15.123:0/3108174427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600ae7e0 0x7fc8600aebf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8600b67a0 tx=0x7fc85802c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2811256b5b1b3e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc85fe00640 1 -- 172.21.15.123:0/3108174427 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858039070 con 0x7fc8600ae7e0 2024-04-25T19:02:47.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc85fe00640 1 -- 172.21.15.123:0/3108174427 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc858034640 con 0x7fc8600ae7e0 2024-04-25T19:02:47.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc85fe00640 1 -- 172.21.15.123:0/3108174427 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858034a10 con 0x7fc8600ae7e0 2024-04-25T19:02:47.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc867e00640 1 -- 172.21.15.123:0/3108174427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600ae7e0 msgr2=0x7fc8600aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc867e00640 1 --2- 172.21.15.123:0/3108174427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600ae7e0 0x7fc8600aebf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8600b67a0 tx=0x7fc85802c370 comp rx=0 tx=0).stop 2024-04-25T19:02:47.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc867e00640 1 -- 172.21.15.123:0/3108174427 shutdown_connections 2024-04-25T19:02:47.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc867e00640 1 --2- 172.21.15.123:0/3108174427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600ae7e0 0x7fc8600aebf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.577+0000 7fc867e00640 1 --2- 172.21.15.123:0/3108174427 >> v2:172.21.15.146:3300/0 conn(0x7fc8600b57f0 0x7fc8600b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.578+0000 7fc867e00640 1 -- 172.21.15.123:0/3108174427 >> 172.21.15.123:0/3108174427 conn(0x7fc8600d1ae0 msgr2=0x7fc8600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.578+0000 7fc867e00640 1 -- 172.21.15.123:0/3108174427 shutdown_connections 2024-04-25T19:02:47.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.578+0000 7fc867e00640 1 -- 172.21.15.123:0/3108174427 wait complete. 2024-04-25T19:02:47.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.580+0000 7fc867e00640 1 Processor -- start 2024-04-25T19:02:47.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.580+0000 7fc867e00640 1 -- start start 2024-04-25T19:02:47.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8600ae7e0 0x7fc86009b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc86015df00 0x7fc8601565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8600b4ff0 con 0x7fc8600b57f0 2024-04-25T19:02:47.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc8600b4d10 con 0x7fc8600ae7e0 2024-04-25T19:02:47.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc867e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc86006f950 con 0x7fc86015df00 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc864a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc865400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8600ae7e0 0x7fc86009b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc864a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39602/0 (socket says 172.21.15.123:39602) 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc864a00640 1 -- 172.21.15.123:0/134132964 learned_addr learned my addr 172.21.15.123:0/134132964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.582+0000 7fc865e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc86015df00 0x7fc8601565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 -- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc86015df00 msgr2=0x7fc8601565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc86015df00 0x7fc8601565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 -- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8600ae7e0 msgr2=0x7fc86009b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8600ae7e0 0x7fc86009b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 -- 172.21.15.123:0/134132964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86009e4b0 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc864a00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc854007e40 tx=0x7fc85400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854014620 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854019070 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854014a00 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.583+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86009e790 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.584+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86009ed50 con 0x7fc8600b57f0 2024-04-25T19:02:47.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.585+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc860076a40 con 0x7fc8600b57f0 2024-04-25T19:02:47.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.586+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc854014ba0 con 0x7fc8600b57f0 2024-04-25T19:02:47.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.587+0000 7fc85d600640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 0x7fc844076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.587+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc854095c10 con 0x7fc8600b57f0 2024-04-25T19:02:47.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.587+0000 7fc865400640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 0x7fc844076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.588+0000 7fc865400640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 0x7fc844076a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc850006fd0 tx=0x7fc850009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.592+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8540601d0 con 0x7fc8600b57f0 2024-04-25T19:02:47.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.793+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc86006e270 con 0x7fc8600b57f0 2024-04-25T19:02:47.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.796+0000 7fc85d600640 1 -- 172.21.15.123:0/134132964 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7fc85405fad0 con 0x7fc8600b57f0 2024-04-25T19:02:47.799 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:47.799 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 msgr2=0x7fc844076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 0x7fc844076a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc850006fd0 tx=0x7fc850009060 comp rx=0 tx=0).stop 2024-04-25T19:02:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 msgr2=0x7fc8601f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc854007e40 tx=0x7fc85400e4b0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 shutdown_connections 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc844074500 0x7fc844076a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc86015df00 0x7fc8601565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8600b57f0 0x7fc8601f5a80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 --2- 172.21.15.123:0/134132964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8600ae7e0 0x7fc86009b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.799+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 >> 172.21.15.123:0/134132964 conn(0x7fc8600d1ae0 msgr2=0x7fc8600684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.800+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 shutdown_connections 2024-04-25T19:02:47.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:47.800+0000 7fc867e00640 1 -- 172.21.15.123:0/134132964 wait complete. 2024-04-25T19:02:48.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T19:02:39.551680+0000', 'modified': '2024-04-25T19:02:44.672679+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6835/1141870368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 1141870368}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 1141870368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4251, 'qdb_cluster': [4251]} 2024-04-25T19:02:48.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T19:02:48.231 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T19:02:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.512+0000 7fc06f200640 1 Processor -- start 2024-04-25T19:02:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.513+0000 7fc06f200640 1 -- start start 2024-04-25T19:02:48.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.515+0000 7fc06f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc07006d580 0x7fc07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.515+0000 7fc06f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07006ded0 con 0x7fc07006fe20 2024-04-25T19:02:48.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.515+0000 7fc06f200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07015f070 con 0x7fc07015e2f0 2024-04-25T19:02:48.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.515+0000 7fc06f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc07015ee10 con 0x7fc07006d580 2024-04-25T19:02:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc06de00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc07006fe20 0x7fc07015b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49408/0 (socket says 172.21.15.123:49408) 2024-04-25T19:02:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc06de00640 1 -- 172.21.15.123:0/548906255 learned_addr learned my addr 172.21.15.123:0/548906255 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc067e00640 1 --2- 172.21.15.123:0/548906255 >> v2:172.21.15.123:3300/0 conn(0x7fc07006d580 0x7fc07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc06d400640 1 -- 172.21.15.123:0/548906255 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104870047 0 0) 0x7fc060001110 con 0x7fc07006fe20 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc06d400640 1 -- 172.21.15.123:0/548906255 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc054003520 con 0x7fc07006fe20 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc067e00640 1 -- 172.21.15.123:0/548906255 >> v1:172.21.15.146:6789/0 conn(0x7fc07015e2f0 legacy=0x7fc07015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc067e00640 1 -- 172.21.15.123:0/548906255 >> v1:172.21.15.123:6789/0 conn(0x7fc07006fe20 legacy=0x7fc07015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.516+0000 7fc067e00640 1 -- 172.21.15.123:0/548906255 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07006f340 con 0x7fc07006d580 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.517+0000 7fc067e00640 1 --2- 172.21.15.123:0/548906255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006d580 0x7fc07006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc05802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582bd008e8201ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06d400640 1 -- 172.21.15.123:0/548906255 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058037070 con 0x7fc07006d580 2024-04-25T19:02:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06d400640 1 -- 172.21.15.123:0/548906255 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05802ae50 con 0x7fc07006d580 2024-04-25T19:02:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06d400640 1 -- 172.21.15.123:0/548906255 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058032640 con 0x7fc07006d580 2024-04-25T19:02:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06f200640 1 -- 172.21.15.123:0/548906255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006d580 msgr2=0x7fc07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06f200640 1 --2- 172.21.15.123:0/548906255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006d580 0x7fc07006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc05802a140 comp rx=0 tx=0).stop 2024-04-25T19:02:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06f200640 1 -- 172.21.15.123:0/548906255 shutdown_connections 2024-04-25T19:02:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.518+0000 7fc06f200640 1 --2- 172.21.15.123:0/548906255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006d580 0x7fc07006d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.519+0000 7fc06f200640 1 -- 172.21.15.123:0/548906255 >> 172.21.15.123:0/548906255 conn(0x7fc070073200 msgr2=0x7fc070073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.519+0000 7fc06f200640 1 -- 172.21.15.123:0/548906255 shutdown_connections 2024-04-25T19:02:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.519+0000 7fc06f200640 1 -- 172.21.15.123:0/548906255 wait complete. 2024-04-25T19:02:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.521+0000 7fc06f200640 1 Processor -- start 2024-04-25T19:02:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.521+0000 7fc06f200640 1 -- start start 2024-04-25T19:02:48.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.522+0000 7fc06f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc07006d580 0x7fc07013bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.522+0000 7fc06f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.522+0000 7fc06f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc07015e2f0 0x7fc0700cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc06f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc070168c70 con 0x7fc07006fe20 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc06f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc07006f900 con 0x7fc07006d580 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc06f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc070168df0 con 0x7fc07015e2f0 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39610/0 (socket says 172.21.15.123:39610) 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 -- 172.21.15.123:0/4098555754 learned_addr learned my addr 172.21.15.123:0/4098555754 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 -- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc07015e2f0 msgr2=0x7fc0700cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc07015e2f0 0x7fc0700cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 -- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc07006d580 msgr2=0x7fc07013bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc07006d580 0x7fc07013bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 -- 172.21.15.123:0/4098555754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07015c790 con 0x7fc07006fe20 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.523+0000 7fc067e00640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc0580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.524+0000 7fc066000640 1 -- 172.21.15.123:0/4098555754 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058042020 con 0x7fc07006fe20 2024-04-25T19:02:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.524+0000 7fc066000640 1 -- 172.21.15.123:0/4098555754 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0580044c0 con 0x7fc07006fe20 2024-04-25T19:02:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.524+0000 7fc066000640 1 -- 172.21.15.123:0/4098555754 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058037040 con 0x7fc07006fe20 2024-04-25T19:02:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.524+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07015ca70 con 0x7fc07006fe20 2024-04-25T19:02:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.524+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07015d030 con 0x7fc07006fe20 2024-04-25T19:02:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.527+0000 7fc066000640 1 -- 172.21.15.123:0/4098555754 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc058045020 con 0x7fc07006fe20 2024-04-25T19:02:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.528+0000 7fc066000640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 0x7fc04c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.528+0000 7fc066000640 1 -- 172.21.15.123:0/4098555754 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc0580b5f10 con 0x7fc07006fe20 2024-04-25T19:02:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.528+0000 7fc06de00640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 0x7fc04c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.528+0000 7fc07b9b4740 1 Processor -- start 2024-04-25T19:02:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.528+0000 7fc07b9b4740 1 -- start start 2024-04-25T19:02:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.529+0000 7fc06de00640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 0x7fc04c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc060000f60 tx=0x7fc060001110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e321270 0x55595e6ee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e708f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55595e627600 con 0x55595e300590 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55595e5c4100 con 0x55595e321270 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55595e6235c0 con 0x55595e2f83b0 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc06e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc06e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39622/0 (socket says 172.21.15.123:39622) 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc06e800640 1 -- 172.21.15.123:0/505776753 learned_addr learned my addr 172.21.15.123:0/505776753 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc06de00640 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e321270 0x55595e6ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.530+0000 7fc067e00640 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e708f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 -- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 msgr2=0x55595e708f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e708f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 -- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e321270 msgr2=0x55595e6ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e321270 0x55595e6ee210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 -- 172.21.15.123:0/505776753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595e3392d0 con 0x55595e300590 2024-04-25T19:02:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc06e800640 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc068009d00 tx=0x7fc06800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4914ed57558171b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc05e600640 1 -- 172.21.15.123:0/505776753 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800bd40 con 0x55595e300590 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc05e600640 1 -- 172.21.15.123:0/505776753 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc068003970 con 0x55595e300590 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.531+0000 7fc05e600640 1 -- 172.21.15.123:0/505776753 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068003b60 con 0x55595e300590 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 -- 172.21.15.123:0/505776753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 msgr2=0x55595e709480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc068009d00 tx=0x7fc06800b420 comp rx=0 tx=0).stop 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 -- 172.21.15.123:0/505776753 shutdown_connections 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e300590 0x55595e709480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e708f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/505776753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e321270 0x55595e6ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 -- 172.21.15.123:0/505776753 >> 172.21.15.123:0/505776753 conn(0x55595e3153c0 msgr2=0x55595e6c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 -- 172.21.15.123:0/505776753 shutdown_connections 2024-04-25T19:02:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.532+0000 7fc07b9b4740 1 -- 172.21.15.123:0/505776753 wait complete. 2024-04-25T19:02:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.533+0000 7fc07b9b4740 1 -- start start 2024-04-25T19:02:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.534+0000 7fc07b9b4740 1 Processor -- start 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e6f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55595e5aef70 con 0x55595e2f83b0 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55595e706c50 con 0x55595e300590 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55595e6c2920 con 0x55595e321270 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.535+0000 7fc07b9b4740 10 client.0 _ll_register_callbacks cb 0x55595e61d480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39630/0 (socket says 172.21.15.123:39630) 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 -- 172.21.15.123:0/3551331111 learned_addr learned my addr 172.21.15.123:0/3551331111 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc067e00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 -- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 msgr2=0x55595e6f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06e800640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e6f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e6f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 -- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 msgr2=0x55595e6e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6e9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 -- 172.21.15.123:0/3551331111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595e4036b0 con 0x55595e2f83b0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc06de00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc060007680 tx=0x7fc060000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c141d77f0e519e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.536+0000 7fc052a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060005e60 con 0x55595e2f83b0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc068003970 con 0x55595e2f83b0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc060023e00 con 0x55595e2f83b0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fc060016b50 con 0x55595e2f83b0 2024-04-25T19:02:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 10 client.4392 fetch_fsmap learned FSMap version 7 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55595e6e7630 con 0x55595e2f83b0 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc068003970 con 0x55595e2f83b0 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fc06002a050 con 0x55595e2f83b0 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 10 client.4392 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 20 client.4392 populate_metadata read hostname 'smithi123' 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e2dfe40 0x55595e704b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 4 client.4392 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.537+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 --> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595e635210 con 0x55595e2dfe40 2024-04-25T19:02:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.538+0000 7fc067e00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e2dfe40 0x55595e704b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.539+0000 7fc067e00640 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e2dfe40 0x55595e704b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f434dbada436d8 server_cookie=f6e1f07d7b638c27 in_seq=0 out_seq=0 2024-04-25T19:02:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.539+0000 7fc052a00640 10 client.4392 ms_handle_connect on v2:172.21.15.146:6834/1141870368 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.540+0000 7fc052a00640 1 -- 172.21.15.123:0/3551331111 <== mds.0 v2:172.21.15.146:6834/1141870368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55595e635210 con 0x55595e2dfe40 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.540+0000 7fc052a00640 10 client.4392 handle_command_reply: tid=0 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 client.4392 shutdown 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 msgr2=0x55595e6e9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc060007680 tx=0x7fc060000ed0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 shutdown_connections 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e2dfe40 0x55595e704b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e6f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/3551331111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6e9780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 >> 172.21.15.123:0/3551331111 conn(0x55595e3153c0 msgr2=0x55595e6c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc052a00640 0 client.4392 ms_handle_reset on v2:172.21.15.146:6834/1141870368 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 shutdown_connections 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 1 -- 172.21.15.123:0/3551331111 wait complete. 2024-04-25T19:02:48.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.541+0000 7fc07b9b4740 20 client.4392 trim_cache size 0 max 16384 2024-04-25T19:02:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.562+0000 7fc07b9b4740 1 Processor -- start 2024-04-25T19:02:48.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.562+0000 7fc07b9b4740 1 -- start start 2024-04-25T19:02:48.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e709020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e709560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc070203210 con 0x55595e2f83b0 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc070202570 con 0x55595e300590 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.563+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc070202bc0 con 0x55595e321270 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39644/0 (socket says 172.21.15.123:39644) 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 -- 172.21.15.123:0/175201043 learned_addr learned my addr 172.21.15.123:0/175201043 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 -- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 msgr2=0x55595e709560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e709560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc067e00640 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e709020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 -- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 msgr2=0x55595e709020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e709020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 -- 172.21.15.123:0/175201043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595e6c18c0 con 0x55595e2f83b0 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc06de00640 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0600013d0 tx=0x7fc06001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45f87a6060af4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.564+0000 7fc05dc00640 1 -- 172.21.15.123:0/175201043 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06002c470 con 0x55595e2f83b0 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc05dc00640 1 -- 172.21.15.123:0/175201043 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc060031020 con 0x55595e2f83b0 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc05dc00640 1 -- 172.21.15.123:0/175201043 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06002c830 con 0x55595e2f83b0 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 -- 172.21.15.123:0/175201043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 msgr2=0x55595e6c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0600013d0 tx=0x7fc06001a000 comp rx=0 tx=0).stop 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 -- 172.21.15.123:0/175201043 shutdown_connections 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e321270 0x55595e709560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e709020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/175201043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e2f83b0 0x55595e6c4d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 -- 172.21.15.123:0/175201043 >> 172.21.15.123:0/175201043 conn(0x55595e3153c0 msgr2=0x55595e6beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.565+0000 7fc07b9b4740 1 -- 172.21.15.123:0/175201043 shutdown_connections 2024-04-25T19:02:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.566+0000 7fc07b9b4740 1 -- 172.21.15.123:0/175201043 wait complete. 2024-04-25T19:02:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.566+0000 7fc07b9b4740 1 -- start start 2024-04-25T19:02:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.567+0000 7fc07b9b4740 1 Processor -- start 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e785770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55595e6f06b0 con 0x55595e321270 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55595e6e7aa0 con 0x55595e300590 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55595e6f0c60 con 0x55595e2f83b0 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.568+0000 7fc07b9b4740 10 client.0 _ll_register_callbacks cb 0x7fc060005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39656/0 (socket says 172.21.15.123:39656) 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 -- 172.21.15.123:0/296510436 learned_addr learned my addr 172.21.15.123:0/296510436 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06de00640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e785770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc067e00640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 -- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 msgr2=0x55595e785770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e785770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 -- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 msgr2=0x55595e6f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 -- 172.21.15.123:0/296510436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595e6c4910 con 0x55595e321270 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc06e800640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b460 tx=0x7fc068004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68db557e8c2bbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.569+0000 7fc052a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068021070 con 0x55595e321270 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc068004250 con 0x55595e321270 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc060031020 con 0x55595e321270 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fc068004490 con 0x55595e321270 2024-04-25T19:02:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 10 client.4398 fetch_fsmap learned FSMap version 7 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55595e786f90 con 0x55595e321270 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc060031020 con 0x55595e321270 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fc068011070 con 0x55595e321270 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 10 client.4398 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 20 client.4398 populate_metadata read hostname 'smithi123' 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e6e6770 0x55595e785f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 4 client.4398 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.570+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 --> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] -- command(tid 0: {"prefix": "status"}) -- 0x55595e62f080 con 0x55595e6e6770 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.571+0000 7fc06de00640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e6e6770 0x55595e785f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.572+0000 7fc06de00640 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e6e6770 0x55595e785f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a83a894a9536755 server_cookie=3a44b3baffd5641f in_seq=0 out_seq=0 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.572+0000 7fc052a00640 10 client.4398 ms_handle_connect on v2:172.21.15.146:6834/1141870368 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.572+0000 7fc052a00640 1 -- 172.21.15.123:0/296510436 <== mds.0 v2:172.21.15.146:6834/1141870368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55595e62f080 con 0x55595e6e6770 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.572+0000 7fc052a00640 10 client.4398 handle_command_reply: tid=0 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.572+0000 7fc07b9b4740 1 client.4398 shutdown 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 msgr2=0x55595e6f1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b460 tx=0x7fc068004a70 comp rx=0 tx=0).stop 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 shutdown_connections 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55595e6e6770 0x55595e785f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55595e321270 0x55595e6f1ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55595e300590 0x55595e6f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 --2- 172.21.15.123:0/296510436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55595e2f83b0 0x55595e785770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 >> 172.21.15.123:0/296510436 conn(0x55595e3153c0 msgr2=0x55595e6beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc052a00640 0 client.4398 ms_handle_reset on v2:172.21.15.146:6834/1141870368 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 shutdown_connections 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 1 -- 172.21.15.123:0/296510436 wait complete. 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.573+0000 7fc07b9b4740 20 client.4398 trim_cache size 0 max 16384 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "a2d72801-e932-4284-9776-fce727cd5af1", 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T19:02:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4251, 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 8.5138479419999999, 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 7, 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 26, 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 14.113088143000001 2024-04-25T19:02:48.578 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:02:48.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 msgr2=0x7fc04c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 0x7fc04c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc060000f60 tx=0x7fc060001110 comp rx=0 tx=0).stop 2024-04-25T19:02:48.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 msgr2=0x7fc0700cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc0580046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 shutdown_connections 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc04c0744b0 0x7fc04c0769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc07015e2f0 0x7fc0700cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07006fe20 0x7fc0700cd830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 --2- 172.21.15.123:0/4098555754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc07006d580 0x7fc07013bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 >> 172.21.15.123:0/4098555754 conn(0x7fc070073200 msgr2=0x7fc07015fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 shutdown_connections 2024-04-25T19:02:48.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.574+0000 7fc06f200640 1 -- 172.21.15.123:0/4098555754 wait complete. 2024-04-25T19:02:48.602 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:02:48.602 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:48.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:02:48.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.869+0000 7f939c600640 1 Processor -- start 2024-04-25T19:02:48.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.869+0000 7f939c600640 1 -- start start 2024-04-25T19:02:48.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.871+0000 7f939c600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f939406f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.871+0000 7f939c600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940b4750 con 0x7f93940b5f00 2024-04-25T19:02:48.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.871+0000 7f939c600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940b4a30 con 0x7f939406f780 2024-04-25T19:02:48.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.871+0000 7f939c600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f939415e310 con 0x7f939415df00 2024-04-25T19:02:48.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.872+0000 7f939a600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f93940b5f00 0x7f93940ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49412/0 (socket says 172.21.15.123:49412) 2024-04-25T19:02:48.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.872+0000 7f939a600640 1 -- 172.21.15.123:0/297325294 learned_addr learned my addr 172.21.15.123:0/297325294 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.872+0000 7f9399c00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f939406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.872+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5322883 0 0) 0x7f93940b4750 con 0x7f93940b5f00 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.872+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9378003520 con 0x7f93940b5f00 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1809096066 0 0) 0x7f9378003520 con 0x7f93940b5f00 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 msgr2=0x7f939406f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f939406f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 >> v1:172.21.15.146:6789/0 conn(0x7f939406f780 legacy=0x7f93940b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940b4d10 con 0x7f93940b5f00 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f938c001670 con 0x7f93940b5f00 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 >> v1:172.21.15.123:6789/0 conn(0x7f93940b5f00 legacy=0x7f93940ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93780039e0 0x7f9378003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f9378004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9378004d40 0x7f93780051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f939415e310 con 0x7f93780039e0 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f937800de00 con 0x7f9378004d40 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f937800e000 con 0x7f939415df00 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.873+0000 7f9399c00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93780039e0 0x7f9378003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f939a600640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9378004d40 0x7f93780051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399200640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f9378004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399c00640 1 -- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 msgr2=0x7f9378004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399c00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f9378004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399c00640 1 -- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9378004d40 msgr2=0x7f93780051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399c00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9378004d40 0x7f93780051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399c00640 1 -- 172.21.15.123:0/297325294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9378003520 con 0x7f93780039e0 2024-04-25T19:02:48.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f9399200640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939415df00 0x7f9378004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:48.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.874+0000 7f939a600640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9378004d40 0x7f93780051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:48.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.875+0000 7f9399c00640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93780039e0 0x7f9378003e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9384009fc0 tx=0x7f9378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f65485d67513292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.875+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384034020 con 0x7f93780039e0 2024-04-25T19:02:48.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.875+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840305f0 con 0x7f93780039e0 2024-04-25T19:02:48.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.875+0000 7f9393e00640 1 -- 172.21.15.123:0/297325294 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384009c10 con 0x7f93780039e0 2024-04-25T19:02:48.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 -- 172.21.15.123:0/297325294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93780039e0 msgr2=0x7f9378003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 --2- 172.21.15.123:0/297325294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93780039e0 0x7f9378003e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9384009fc0 tx=0x7f9378005830 comp rx=0 tx=0).stop 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939a600640 1 -- 172.21.15.123:0/297325294 reap_dead start 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 -- 172.21.15.123:0/297325294 shutdown_connections 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 -- 172.21.15.123:0/297325294 >> 172.21.15.123:0/297325294 conn(0x7f93940d1ae0 msgr2=0x7f93940d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 -- 172.21.15.123:0/297325294 shutdown_connections 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.876+0000 7f939c600640 1 -- 172.21.15.123:0/297325294 wait complete. 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.878+0000 7f939c600640 1 Processor -- start 2024-04-25T19:02:48.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.878+0000 7f939c600640 1 -- start start 2024-04-25T19:02:48.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93780039e0 0x7f93940cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939414afe0 0x7f93941e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93940aeef0 con 0x7f939414b3f0 2024-04-25T19:02:48.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93940b5360 con 0x7f93780039e0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939c600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93940af070 con 0x7f939414afe0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39682/0 (socket says 172.21.15.123:39682) 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.880+0000 7f939a600640 1 -- 172.21.15.123:0/3895530316 learned_addr learned my addr 172.21.15.123:0/3895530316 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f9399c00640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93780039e0 0x7f93940cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f9399200640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939414afe0 0x7f93941e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 -- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939414afe0 msgr2=0x7f93941e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939414afe0 0x7f93941e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 -- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93780039e0 msgr2=0x7f93940cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93780039e0 0x7f93940cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 -- 172.21.15.123:0/3895530316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940d06d0 con 0x7f939414b3f0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f939a600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f93940745d0 tx=0x7f938c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.881+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c007440 con 0x7f939414b3f0 2024-04-25T19:02:48.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.882+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c0075e0 con 0x7f939414b3f0 2024-04-25T19:02:48.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.882+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0049e0 con 0x7f939414b3f0 2024-04-25T19:02:48.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.882+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93940d09b0 con 0x7f939414b3f0 2024-04-25T19:02:48.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.882+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9394155a10 con 0x7f939414b3f0 2024-04-25T19:02:48.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.883+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93940d0440 con 0x7f939414b3f0 2024-04-25T19:02:48.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.884+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f938c004b80 con 0x7f939414b3f0 2024-04-25T19:02:48.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.886+0000 7f9391600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 0x7f93740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.886+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f938c094db0 con 0x7f939414b3f0 2024-04-25T19:02:48.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.886+0000 7f9399c00640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 0x7f93740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.887+0000 7f9399c00640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 0x7f93740769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9384035000 tx=0x7f938402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:48.891+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f938c099070 con 0x7f939414b3f0 2024-04-25T19:02:49.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.131+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9394076a40 con 0x7f939414b3f0 2024-04-25T19:02:49.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.132+0000 7f9391600640 1 -- 172.21.15.123:0/3895530316 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f938c01a060 con 0x7f939414b3f0 2024-04-25T19:02:49.134 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 msgr2=0x7f93740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 0x7f93740769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9384035000 tx=0x7f938402f000 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 msgr2=0x7f93941e6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f93940745d0 tx=0x7f938c00d890 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 shutdown_connections 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93740744e0 0x7f93740769f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f939414b3f0 0x7f93941e6340 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939414afe0 0x7f93941e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 --2- 172.21.15.123:0/3895530316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93780039e0 0x7f93940cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:49.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 >> 172.21.15.123:0/3895530316 conn(0x7f93940d1ae0 msgr2=0x7f9394153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 shutdown_connections 2024-04-25T19:02:49.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49.135+0000 7f939c600640 1 -- 172.21.15.123:0/3895530316 wait complete. 2024-04-25T19:02:49.162 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T19:02:49.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:02:49.162 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:02:49.163 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T19:02:49.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:49.163 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:02:49.215 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42628sec preferred_lft 42628sec 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T19:02:49.216 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:49.217 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T19:02:49.217 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:02:49.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:49.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:49.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:49.689 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:02:49.758 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T19:02:49.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:49.761 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T19:02:49.811 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T19:02:49.811 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:02:49.812 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:02:49.812 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:02:49.812 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:02:49.812 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:49.812 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:02:49.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:02:49.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:02:49.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:02:49.813 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:02:49.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:50.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:50.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:50.540 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:02:50.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:50.588 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T19:02:50.637 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:50.638 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:02:50.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:50.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T19:02:50.638 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:02:50.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:50.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:50.822 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T19:02:50.822 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:50.822 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:02:50.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T19:02:50.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T19:02:50.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T19:02:50.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T19:02:50.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T19:02:50.823 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:02:50.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:51.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:51.106 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:51.106 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:02:51.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T19:02:51.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T19:02:51.106 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:02:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:51.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:51.299 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T19:02:51.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:02:51.299 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:51.343 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T19:02:51.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:02:51.344 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:51.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T19:02:51.517 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T19:02:51.565 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T19:02:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T19:02:51.567 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T19:02:51.568 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T19:02:51.568 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T19:02:51.568 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:51.568 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:51.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:51.569 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T19:02:51.614 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T19:02:51.615 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T19:02:51.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T19:02:51.735 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T19:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T19:02:51.784 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:51.785 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:51.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:51.786 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T19:02:51.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:2024-04-25T19:02:51.927+0000 7f2e2cd65480 -1 init, newargv = 0x55a811459ea0 newargc=15 2024-04-25T19:02:51.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40205]: starting ceph client 2024-04-25T19:02:51.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40205]: starting fuse 2024-04-25T19:02:52.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:02:52.708 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:02:52.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-25T19:02:53.058 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-25T19:02:53.111 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-25T19:02:53.112 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-25T19:02:53.113 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:53.114 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:02:53.114 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:53.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.115 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-25T19:02:53.165 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-25T19:02:53.165 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-25T19:02:53.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-25T19:02:53.166 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.admin")) 2024-04-25T19:02:53.167 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-25T19:02:53.333 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.admin.40205.asok 2024-04-25T19:02:53.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:53.352 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.40205.asok 2024-04-25T19:02:53.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.40205.asok status 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "admin", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "40205", 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-25T19:02:53.645 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4407, 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4407 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4078190559 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-25T19:02:53.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4078190559 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4407 192.168.0.1:0/4078190559", 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/4078190559", 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 7, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 26, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-25T19:02:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:02:53.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.666 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-25T19:02:53.666 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:53.716 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-25T19:02:53.717 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:53.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:02:53.788 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T19:02:53.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:02:53.926 INFO:teuthology.orchestra.run.smithi123.stdout:# file: . 2024-04-25T19:02:53.927 INFO:teuthology.orchestra.run.smithi123.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T19:02:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:53.936 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:53.936 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:02:53.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:54.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.281+0000 7f93b2400640 1 Processor -- start 2024-04-25T19:02:54.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.282+0000 7f93b2400640 1 -- start start 2024-04-25T19:02:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b2400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93ac06dc90 0x7f93ac0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b2400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f93ac146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b2400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ac1456e0 con 0x7f93ac1450d0 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b2400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f93ac1462f0 con 0x7f93ac06dc90 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b2400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac1459c0 con 0x7f93ac0b0af0 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b1000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f93ac1450d0 0x7f93ac06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49416/0 (socket says 172.21.15.123:49416) 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.284+0000 7f93b1000640 1 -- 172.21.15.123:0/1670080258 learned_addr learned my addr 172.21.15.123:0/1670080258 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93abe00640 1 --2- 172.21.15.123:0/1670080258 >> v2:172.21.15.146:3300/0 conn(0x7f93ac06dc90 0x7f93ac0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93b1a00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f93ac146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577844968 0 0) 0x7f93ac1456e0 con 0x7f93ac1450d0 2024-04-25T19:02:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9394003520 con 0x7f93ac1450d0 2024-04-25T19:02:54.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1485699587 0 0) 0x7f9394003520 con 0x7f93ac1450d0 2024-04-25T19:02:54.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 msgr2=0x7f93ac146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f93ac146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 >> v2:172.21.15.146:3300/0 conn(0x7f93ac06dc90 msgr2=0x7f93ac0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 --2- 172.21.15.123:0/1670080258 >> v2:172.21.15.146:3300/0 conn(0x7f93ac06dc90 0x7f93ac0b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac145ca0 con 0x7f93ac1450d0 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f93a0001670 con 0x7f93ac1450d0 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 >> v1:172.21.15.123:6789/0 conn(0x7f93ac1450d0 legacy=0x7f93ac06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93940039b0 0x7f9394003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f9394004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.285+0000 7f93ab400640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f9394005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1462f0 con 0x7f9394004800 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1459c0 con 0x7f93940039b0 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f939400b9c0 con 0x7f93ac0b0af0 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f9394004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93b1000640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93940039b0 0x7f9394003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93b1a00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f9394005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 -- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 msgr2=0x7f9394005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac0b0af0 0x7f9394005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 -- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93940039b0 msgr2=0x7f9394003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93940039b0 0x7f9394003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93abe00640 1 -- 172.21.15.123:0/1670080258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9394003520 con 0x7f9394004800 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.286+0000 7f93b1000640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93940039b0 0x7f9394003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.287+0000 7f93abe00640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f9394004c10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f939c009ac0 tx=0x7f9394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3395d63495ed80e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.287+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c036070 con 0x7f9394004800 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c031400 con 0x7f9394004800 2024-04-25T19:02:54.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93ab400640 1 -- 172.21.15.123:0/1670080258 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c009690 con 0x7f9394004800 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93b2400640 1 -- 172.21.15.123:0/1670080258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 msgr2=0x7f9394004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93b2400640 1 --2- 172.21.15.123:0/1670080258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f9394004c10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f939c009ac0 tx=0x7f9394005830 comp rx=0 tx=0).stop 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93b1a00640 1 -- 172.21.15.123:0/1670080258 reap_dead start 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93b2400640 1 -- 172.21.15.123:0/1670080258 shutdown_connections 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.288+0000 7f93b2400640 1 -- 172.21.15.123:0/1670080258 >> 172.21.15.123:0/1670080258 conn(0x7f93ac073200 msgr2=0x7f93ac073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.289+0000 7f93b2400640 1 -- 172.21.15.123:0/1670080258 shutdown_connections 2024-04-25T19:02:54.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.289+0000 7f93b2400640 1 -- 172.21.15.123:0/1670080258 wait complete. 2024-04-25T19:02:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.291+0000 7f93b2400640 1 Processor -- start 2024-04-25T19:02:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.291+0000 7f93b2400640 1 -- start start 2024-04-25T19:02:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 0x7f93ac15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac1616b0 0x7f93ac161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0b1080 con 0x7f9394004800 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0b0f00 con 0x7f93ac142d70 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b2400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac146ad0 con 0x7f93ac1616b0 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39712/0 (socket says 172.21.15.123:39712) 2024-04-25T19:02:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 -- 172.21.15.123:0/968135487 learned_addr learned my addr 172.21.15.123:0/968135487 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1a00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac1616b0 0x7f93ac161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93abe00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 0x7f93ac15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 -- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac1616b0 msgr2=0x7f93ac161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac1616b0 0x7f93ac161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 -- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 msgr2=0x7f93ac15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 0x7f93ac15f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93b1000640 1 -- 172.21.15.123:0/968135487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac143550 con 0x7f9394004800 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.293+0000 7f93abe00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 0x7f93ac15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93b1000640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000fb0 tx=0x7f93a000dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0019070 con 0x7f9394004800 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a00074e0 con 0x7f9394004800 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a00049e0 con 0x7f9394004800 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac160650 con 0x7f9394004800 2024-04-25T19:02:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.294+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac160bd0 con 0x7f9394004800 2024-04-25T19:02:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.295+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac148030 con 0x7f9394004800 2024-04-25T19:02:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.297+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f93a0004b80 con 0x7f9394004800 2024-04-25T19:02:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.298+0000 7f93a8c00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 0x7f93900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.298+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f93a0094fd0 con 0x7f9394004800 2024-04-25T19:02:54.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.298+0000 7f93abe00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 0x7f93900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.299+0000 7f93abe00640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 0x7f93900769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f939c033000 tx=0x7f939c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.308+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93ac148030 con 0x7f9394004800 2024-04-25T19:02:54.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.550+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93ac073a80 con 0x7f9394004800 2024-04-25T19:02:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.552+0000 7f93a8c00640 1 -- 172.21.15.123:0/968135487 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f93a005f380 con 0x7f9394004800 2024-04-25T19:02:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:54.554 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 msgr2=0x7f93900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 0x7f93900769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f939c033000 tx=0x7f939c008030 comp rx=0 tx=0).stop 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 msgr2=0x7f93ac1f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000fb0 tx=0x7f93a000dc50 comp rx=0 tx=0).stop 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 shutdown_connections 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93900744b0 0x7f93900769c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac1616b0 0x7f93ac161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac142d70 0x7f93ac15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 --2- 172.21.15.123:0/968135487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9394004800 0x7f93ac1f5ee0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 >> 172.21.15.123:0/968135487 conn(0x7f93ac073200 msgr2=0x7f93ac0b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 shutdown_connections 2024-04-25T19:02:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.555+0000 7f93b2400640 1 -- 172.21.15.123:0/968135487 wait complete. 2024-04-25T19:02:54.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:02:54.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:02:54.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.842+0000 7f7eb6e00640 1 Processor -- start 2024-04-25T19:02:54.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.843+0000 7f7eb6e00640 1 -- start start 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00d11a0 0x7f7eb006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb006cc00 con 0x7f7eb0163d00 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb006cee0 con 0x7f7eb006dca0 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb006d1c0 con 0x7f7eb00d11a0 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7eb0163d00 0x7f7eb00b90a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49420/0 (socket says 172.21.15.123:49420) 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb6400640 1 -- 172.21.15.123:0/3797010870 learned_addr learned my addr 172.21.15.123:0/3797010870 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.845+0000 7f7eb5000640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00d11a0 0x7f7eb006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518650160 0 0) 0x7f7eb006cc00 con 0x7f7eb0163d00 2024-04-25T19:02:54.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e98003520 con 0x7f7eb0163d00 2024-04-25T19:02:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1307355196 0 0) 0x7f7e98003520 con 0x7f7eb0163d00 2024-04-25T19:02:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00d11a0 msgr2=0x7f7eb006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00d11a0 0x7f7eb006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 >> v1:172.21.15.146:6789/0 conn(0x7f7eb006dca0 legacy=0x7f7eb00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb006d4a0 con 0x7f7eb0163d00 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f7ea80019f0 con 0x7f7eb0163d00 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 >> v1:172.21.15.123:6789/0 conn(0x7f7eb0163d00 legacy=0x7f7eb00b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e980039e0 0x7f7e98003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98004800 0x7f7e98004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e98005270 0x7f7e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb006d1c0 con 0x7f7e98004800 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9800bfb0 con 0x7f7e980039e0 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9800c1b0 con 0x7f7e98005270 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.846+0000 7f7eb5000640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98004800 0x7f7e98004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5a00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e980039e0 0x7f7e98003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb6400640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e98005270 0x7f7e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5000640 1 -- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e98005270 msgr2=0x7f7e9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5000640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e98005270 0x7f7e9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5000640 1 -- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e980039e0 msgr2=0x7f7e98003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5000640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e980039e0 0x7f7e98003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb5000640 1 -- 172.21.15.123:0/3797010870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98003520 con 0x7f7e98004800 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.847+0000 7f7eb6400640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e98005270 0x7f7e9800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.848+0000 7f7eb5000640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98004800 0x7f7e98004c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009ac0 tx=0x7f7e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81788ebe1dea22c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.848+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c036070 con 0x7f7e98004800 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.848+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c031400 con 0x7f7e98004800 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.848+0000 7f7eafe00640 1 -- 172.21.15.123:0/3797010870 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c009690 con 0x7f7e98004800 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 -- 172.21.15.123:0/3797010870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98004800 msgr2=0x7f7e98004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/3797010870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98004800 0x7f7e98004c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009ac0 tx=0x7f7e98005830 comp rx=0 tx=0).stop 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6400640 1 -- 172.21.15.123:0/3797010870 reap_dead start 2024-04-25T19:02:54.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 -- 172.21.15.123:0/3797010870 shutdown_connections 2024-04-25T19:02:54.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 -- 172.21.15.123:0/3797010870 >> 172.21.15.123:0/3797010870 conn(0x7f7eb00d6510 msgr2=0x7f7eb00d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:54.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 -- 172.21.15.123:0/3797010870 shutdown_connections 2024-04-25T19:02:54.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.849+0000 7f7eb6e00640 1 -- 172.21.15.123:0/3797010870 wait complete. 2024-04-25T19:02:54.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.851+0000 7f7eb6e00640 1 Processor -- start 2024-04-25T19:02:54.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.852+0000 7f7eb6e00640 1 -- start start 2024-04-25T19:02:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 0x7f7eb01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00ae7e0 0x7f7eb0159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00d11a0 con 0x7f7eb0162f40 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b89f0 con 0x7f7e98004800 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb6e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb00d1320 con 0x7f7eb00ae7e0 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39730/0 (socket says 172.21.15.123:39730) 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.853+0000 7f7eb5000640 1 -- 172.21.15.123:0/1631123023 learned_addr learned my addr 172.21.15.123:0/1631123023 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5a00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 0x7f7eb01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb6400640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00ae7e0 0x7f7eb0159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 -- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00ae7e0 msgr2=0x7f7eb0159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00ae7e0 0x7f7eb0159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 -- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 msgr2=0x7f7eb01595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 0x7f7eb01595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 -- 172.21.15.123:0/1631123023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0161ea0 con 0x7f7eb0162f40 2024-04-25T19:02:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5a00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 0x7f7eb01595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7eb5000640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c034040 tx=0x7f7e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c040020 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.854+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c003b40 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.855+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c036040 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.855+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb0162180 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.855+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb0162740 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.856+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb0050740 con 0x7f7eb0162f40 2024-04-25T19:02:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.857+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e9c043020 con 0x7f7eb0162f40 2024-04-25T19:02:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.858+0000 7f7ead600640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 0x7f7e900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.858+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7e9c0b57f0 con 0x7f7eb0162f40 2024-04-25T19:02:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.858+0000 7f7eb5a00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 0x7f7e900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:54.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.859+0000 7f7eb5a00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 0x7f7e900769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001630 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:54.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:54.863+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e9c04a070 con 0x7f7eb0162f40 2024-04-25T19:02:55.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.064+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7eb00d0660 con 0x7f7eb0162f40 2024-04-25T19:02:55.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.065+0000 7f7ead600640 1 -- 172.21.15.123:0/1631123023 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f7e9c0044e0 con 0x7f7eb0162f40 2024-04-25T19:02:55.067 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 msgr2=0x7f7e900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 0x7f7e900769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001630 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 msgr2=0x7f7eb00ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c034040 tx=0x7f7e9c004730 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 shutdown_connections 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e900744b0 0x7f7e900769c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb00ae7e0 0x7f7eb0159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb0162f40 0x7f7eb00ae2a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 --2- 172.21.15.123:0/1631123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e98004800 0x7f7eb01595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 >> 172.21.15.123:0/1631123023 conn(0x7f7eb00d6510 msgr2=0x7f7eb00ba1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:55.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 shutdown_connections 2024-04-25T19:02:55.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.068+0000 7f7eb6e00640 1 -- 172.21.15.123:0/1631123023 wait complete. 2024-04-25T19:02:55.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:02:55.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.351+0000 7f72ba200640 1 Processor -- start 2024-04-25T19:02:55.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.351+0000 7f72ba200640 1 -- start start 2024-04-25T19:02:55.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.353+0000 7f72ba200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f72b406dca0 0x7f72b406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.353+0000 7f72ba200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72b40d11a0 0x7f72b40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.353+0000 7f72ba200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b406cc00 con 0x7f72b4163d00 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72ba200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f72b40ba0b0 con 0x7f72b40d11a0 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72ba200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f72b406cee0 con 0x7f72b406dca0 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72b40d11a0 0x7f72b40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72b40d11a0 0x7f72b40b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43166/0 (socket says 172.21.15.123:43166) 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 -- 172.21.15.123:0/3882568386 learned_addr learned my addr 172.21.15.123:0/3882568386 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:55.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b8e00640 1 --2- 172.21.15.123:0/3882568386 >> v2:172.21.15.146:3300/0 conn(0x7f72b406dca0 0x7f72b406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 -- 172.21.15.123:0/3882568386 >> v2:172.21.15.146:3300/0 conn(0x7f72b406dca0 msgr2=0x7f72b406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:55.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 --2- 172.21.15.123:0/3882568386 >> v2:172.21.15.146:3300/0 conn(0x7f72b406dca0 0x7f72b406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 -- 172.21.15.123:0/3882568386 >> v1:172.21.15.146:6789/0 conn(0x7f72b4163d00 legacy=0x7f72b40d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:55.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.354+0000 7f72b9800640 1 -- 172.21.15.123:0/3882568386 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b406d1c0 con 0x7f72b40d11a0 2024-04-25T19:02:55.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.355+0000 7f72b9800640 1 --2- 172.21.15.123:0/3882568386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b40d11a0 0x7f72b40b4c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f72b408e6b0 tx=0x7f72ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab2283fc2197412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.356+0000 7f72b3400640 1 -- 172.21.15.123:0/3882568386 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac036070 con 0x7f72b40d11a0 2024-04-25T19:02:55.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.356+0000 7f72b3400640 1 -- 172.21.15.123:0/3882568386 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac031600 con 0x7f72b40d11a0 2024-04-25T19:02:55.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.356+0000 7f72b3400640 1 -- 172.21.15.123:0/3882568386 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0319d0 con 0x7f72b40d11a0 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 -- 172.21.15.123:0/3882568386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b40d11a0 msgr2=0x7f72b40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 --2- 172.21.15.123:0/3882568386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b40d11a0 0x7f72b40b4c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f72b408e6b0 tx=0x7f72ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 -- 172.21.15.123:0/3882568386 shutdown_connections 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 --2- 172.21.15.123:0/3882568386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b40d11a0 0x7f72b40b4c10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 --2- 172.21.15.123:0/3882568386 >> v2:172.21.15.146:3300/0 conn(0x7f72b406dca0 0x7f72b406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 -- 172.21.15.123:0/3882568386 >> 172.21.15.123:0/3882568386 conn(0x7f72b40d6510 msgr2=0x7f72b40d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 -- 172.21.15.123:0/3882568386 shutdown_connections 2024-04-25T19:02:55.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.357+0000 7f72ba200640 1 -- 172.21.15.123:0/3882568386 wait complete. 2024-04-25T19:02:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.359+0000 7f72ba200640 1 Processor -- start 2024-04-25T19:02:55.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.359+0000 7f72ba200640 1 -- start start 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b406dca0 0x7f72b4150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b4163d00 0x7f72b4162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72b40b89f0 con 0x7f72b4162c70 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72b406d780 con 0x7f72b4163d00 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72ba200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f72b40b90a0 con 0x7f72b406dca0 2024-04-25T19:02:55.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43176/0 (socket says 172.21.15.123:43176) 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 -- 172.21.15.123:0/1741409035 learned_addr learned my addr 172.21.15.123:0/1741409035 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b8e00640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b406dca0 0x7f72b4150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b3e00640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b4163d00 0x7f72b4162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 -- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b406dca0 msgr2=0x7f72b4150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b406dca0 0x7f72b4150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 -- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b4163d00 msgr2=0x7f72b4162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b4163d00 0x7f72b4162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.361+0000 7f72b9800640 1 -- 172.21.15.123:0/1741409035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b4163080 con 0x7f72b4162c70 2024-04-25T19:02:55.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72b9800640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72ac02ace0 tx=0x7f72ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac036050 con 0x7f72b4162c70 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac042070 con 0x7f72b4162c70 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b4160cf0 con 0x7f72b4162c70 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac009510 con 0x7f72b4162c70 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.362+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41612b0 con 0x7f72b4162c70 2024-04-25T19:02:55.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.363+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b4050740 con 0x7f72b4162c70 2024-04-25T19:02:55.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.364+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72ac007bd0 con 0x7f72b4162c70 2024-04-25T19:02:55.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.365+0000 7f72b0c00640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.366+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f72ac0b5ed0 con 0x7f72b4162c70 2024-04-25T19:02:55.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.366+0000 7f72b8e00640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.366+0000 7f72b8e00640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 0x7f72940769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.370+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72ac04b020 con 0x7f72b4162c70 2024-04-25T19:02:55.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.569+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f72b40d0660 con 0x7f72b4162c70 2024-04-25T19:02:55.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.570+0000 7f72b0c00640 1 -- 172.21.15.123:0/1741409035 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f72b4163080 con 0x7f72b4162c70 2024-04-25T19:02:55.572 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.573+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 msgr2=0x7f72940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.573+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 0x7f72940769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4008040 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.573+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 msgr2=0x7f72b40cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.573+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72ac02ace0 tx=0x7f72ac004a40 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 shutdown_connections 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72b4162c70 0x7f72b40cb2c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b4163d00 0x7f72b4162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 --2- 172.21.15.123:0/1741409035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b406dca0 0x7f72b4150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 >> 172.21.15.123:0/1741409035 conn(0x7f72b40d6510 msgr2=0x7f72b40b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:55.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 shutdown_connections 2024-04-25T19:02:55.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.574+0000 7f72ba200640 1 -- 172.21.15.123:0/1741409035 wait complete. 2024-04-25T19:02:55.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:02:55.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.865+0000 7f048f200640 1 Processor -- start 2024-04-25T19:02:55.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.865+0000 7f048f200640 1 -- start start 2024-04-25T19:02:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048f200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f04900b08c0 0x7f04900cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 0x7f04900cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048f200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04900cfe50 con 0x7f049006fcd0 2024-04-25T19:02:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048f200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f04900707f0 con 0x7f04900b08c0 2024-04-25T19:02:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0490070a20 con 0x7f04900cf500 2024-04-25T19:02:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 0x7f04900cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.868+0000 7f048de00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f04900b08c0 0x7f04900cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 0x7f04900cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:48026/0 (socket says 172.21.15.123:48026) 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048d400640 1 -- 172.21.15.123:0/3964153667 learned_addr learned my addr 172.21.15.123:0/3964153667 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048de00640 1 -- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 msgr2=0x7f04900cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048de00640 1 --2- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 0x7f04900cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048de00640 1 -- 172.21.15.123:0/3964153667 >> v1:172.21.15.146:6789/0 conn(0x7f049006fcd0 legacy=0x7f04900700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.869+0000 7f048de00640 1 -- 172.21.15.123:0/3964153667 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049006c780 con 0x7f04900b08c0 2024-04-25T19:02:55.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.871+0000 7f048de00640 1 --2- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04900b08c0 0x7f04900cd830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04800097a0 tx=0x7f048002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fccff50bfe85ceed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.871+0000 7f048ca00640 1 -- 172.21.15.123:0/3964153667 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480037070 con 0x7f04900b08c0 2024-04-25T19:02:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.871+0000 7f048ca00640 1 -- 172.21.15.123:0/3964153667 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0480032440 con 0x7f04900b08c0 2024-04-25T19:02:55.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.871+0000 7f048ca00640 1 -- 172.21.15.123:0/3964153667 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480032810 con 0x7f04900b08c0 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 -- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04900b08c0 msgr2=0x7f04900cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 --2- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04900b08c0 0x7f04900cd830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04800097a0 tx=0x7f048002a180 comp rx=0 tx=0).stop 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 -- 172.21.15.123:0/3964153667 shutdown_connections 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 --2- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900cf500 0x7f04900cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 --2- 172.21.15.123:0/3964153667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04900b08c0 0x7f04900cd830 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 -- 172.21.15.123:0/3964153667 >> 172.21.15.123:0/3964153667 conn(0x7f04900d19f0 msgr2=0x7f04900d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 -- 172.21.15.123:0/3964153667 shutdown_connections 2024-04-25T19:02:55.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.872+0000 7f048f200640 1 -- 172.21.15.123:0/3964153667 wait complete. 2024-04-25T19:02:55.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.875+0000 7f048f200640 1 Processor -- start 2024-04-25T19:02:55.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.876+0000 7f048f200640 1 -- start start 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 0x7f04900b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900b08c0 0x7f04901e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f049006d260 con 0x7f04900cf500 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f049006cd40 con 0x7f049006fcd0 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f049006d3e0 con 0x7f04900b08c0 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43192/0 (socket says 172.21.15.123:43192) 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048e800640 1 -- 172.21.15.123:0/1635815970 learned_addr learned my addr 172.21.15.123:0/1635815970 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048d400640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900b08c0 0x7f04901e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.878+0000 7f048de00640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 0x7f04900b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 -- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900b08c0 msgr2=0x7f04901e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900b08c0 0x7f04901e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 -- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 msgr2=0x7f04900b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 0x7f04900b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 -- 172.21.15.123:0/1635815970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049015d700 con 0x7f04900cf500 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048de00640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 0x7f04900b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048e800640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f048800c6e0 tx=0x7f048800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488003b70 con 0x7f04900cf500 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488003d10 con 0x7f04900cf500 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04900b58f0 con 0x7f04900cf500 2024-04-25T19:02:55.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488005310 con 0x7f04900cf500 2024-04-25T19:02:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.879+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0490147360 con 0x7f04900cf500 2024-04-25T19:02:55.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.880+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04901475c0 con 0x7f04900cf500 2024-04-25T19:02:55.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.882+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f04880054b0 con 0x7f04900cf500 2024-04-25T19:02:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.883+0000 7f0486000640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 0x7f04700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.883+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0488095380 con 0x7f04900cf500 2024-04-25T19:02:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.883+0000 7f048de00640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 0x7f04700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:55.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.884+0000 7f048de00640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 0x7f04700769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0480034000 tx=0x7f048002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:55.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:55.888+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04900cfe50 con 0x7f04900cf500 2024-04-25T19:02:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.078+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0490050740 con 0x7f04900cf500 2024-04-25T19:02:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.079+0000 7f0486000640 1 -- 172.21.15.123:0/1635815970 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f048805f730 con 0x7f04900cf500 2024-04-25T19:02:56.081 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:02:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 msgr2=0x7f04700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 0x7f04700769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0480034000 tx=0x7f048002e000 comp rx=0 tx=0).stop 2024-04-25T19:02:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 msgr2=0x7f0490148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f048800c6e0 tx=0x7f048800cba0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 shutdown_connections 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04700744e0 0x7f04700769f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.081+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04900cf500 0x7f0490148bd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.082+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04900b08c0 0x7f04901e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.082+0000 7f048f200640 1 --2- 172.21.15.123:0/1635815970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049006fcd0 0x7f04900b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.082+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 >> 172.21.15.123:0/1635815970 conn(0x7f04900d19f0 msgr2=0x7f0490161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.082+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 shutdown_connections 2024-04-25T19:02:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.082+0000 7f048f200640 1 -- 172.21.15.123:0/1635815970 wait complete. 2024-04-25T19:02:56.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:02:56.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.375+0000 7fc625800640 1 Processor -- start 2024-04-25T19:02:56.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.375+0000 7fc625800640 1 -- start start 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.377+0000 7fc625800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc6200cfff0 0x7fc62015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.377+0000 7fc625800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc62006dc90 0x7fc6200cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.377+0000 7fc625800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62015e2e0 con 0x7fc620143040 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.377+0000 7fc625800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc6200d1200 con 0x7fc62006dc90 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.377+0000 7fc625800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc6200d05f0 con 0x7fc6200cfff0 2024-04-25T19:02:56.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc62006dc90 0x7fc6200cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc62006dc90 0x7fc6200cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43204/0 (socket says 172.21.15.123:43204) 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 -- 172.21.15.123:0/96398619 learned_addr learned my addr 172.21.15.123:0/96398619 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc61fe00640 1 --2- 172.21.15.123:0/96398619 >> v2:172.21.15.146:3300/0 conn(0x7fc6200cfff0 0x7fc62015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 -- 172.21.15.123:0/96398619 >> v2:172.21.15.146:3300/0 conn(0x7fc6200cfff0 msgr2=0x7fc62015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 --2- 172.21.15.123:0/96398619 >> v2:172.21.15.146:3300/0 conn(0x7fc6200cfff0 0x7fc62015dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 -- 172.21.15.123:0/96398619 >> v1:172.21.15.123:6789/0 conn(0x7fc620143040 legacy=0x7fc62006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.378+0000 7fc624e00640 1 -- 172.21.15.123:0/96398619 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6200d08d0 con 0x7fc62006dc90 2024-04-25T19:02:56.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.379+0000 7fc624e00640 1 --2- 172.21.15.123:0/96398619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc62006dc90 0x7fc6200cd770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc620072580 tx=0x7fc61802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8369bed319fe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc61f400640 1 -- 172.21.15.123:0/96398619 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618037070 con 0x7fc62006dc90 2024-04-25T19:02:56.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc61f400640 1 -- 172.21.15.123:0/96398619 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618032440 con 0x7fc62006dc90 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc61f400640 1 -- 172.21.15.123:0/96398619 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618032810 con 0x7fc62006dc90 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 -- 172.21.15.123:0/96398619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc62006dc90 msgr2=0x7fc6200cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 --2- 172.21.15.123:0/96398619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc62006dc90 0x7fc6200cd770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc620072580 tx=0x7fc61802a1f0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 -- 172.21.15.123:0/96398619 shutdown_connections 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 --2- 172.21.15.123:0/96398619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc62006dc90 0x7fc6200cd770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 --2- 172.21.15.123:0/96398619 >> v2:172.21.15.146:3300/0 conn(0x7fc6200cfff0 0x7fc62015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.380+0000 7fc625800640 1 -- 172.21.15.123:0/96398619 >> 172.21.15.123:0/96398619 conn(0x7fc620073090 msgr2=0x7fc6200734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.381+0000 7fc625800640 1 -- 172.21.15.123:0/96398619 shutdown_connections 2024-04-25T19:02:56.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.381+0000 7fc625800640 1 -- 172.21.15.123:0/96398619 wait complete. 2024-04-25T19:02:56.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.383+0000 7fc625800640 1 Processor -- start 2024-04-25T19:02:56.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.383+0000 7fc625800640 1 -- start start 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.384+0000 7fc625800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc625800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc620143040 0x7fc620139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc625800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620139570 0x7fc62013af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc625800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc620143450 con 0x7fc6200cfff0 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc625800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200d0e90 con 0x7fc620139570 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc625800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc620139d50 con 0x7fc620143040 2024-04-25T19:02:56.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43208/0 (socket says 172.21.15.123:43208) 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 -- 172.21.15.123:0/4273872014 learned_addr learned my addr 172.21.15.123:0/4273872014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 -- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc620143040 msgr2=0x7fc620139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc620143040 0x7fc620139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 -- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620139570 msgr2=0x7fc62013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620139570 0x7fc62013af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 -- 172.21.15.123:0/4273872014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201f7110 con 0x7fc6200cfff0 2024-04-25T19:02:56.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.385+0000 7fc61fe00640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc61000d600 tx=0x7fc61000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.386+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610004240 con 0x7fc6200cfff0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.386+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6100043e0 con 0x7fc6200cfff0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.386+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610003830 con 0x7fc6200cfff0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.386+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201f6e30 con 0x7fc6200cfff0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.386+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6200b4660 con 0x7fc6200cfff0 2024-04-25T19:02:56.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.387+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620073e60 con 0x7fc6200cfff0 2024-04-25T19:02:56.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.388+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc6100039d0 con 0x7fc6200cfff0 2024-04-25T19:02:56.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.389+0000 7fc61cc00640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 0x7fc5fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.389+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc6100958d0 con 0x7fc6200cfff0 2024-04-25T19:02:56.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.390+0000 7fc617400640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 0x7fc5fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.390+0000 7fc617400640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 0x7fc5fc0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6080012e0 tx=0x7fc608007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.394+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc620073e60 con 0x7fc6200cfff0 2024-04-25T19:02:56.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.587+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc620050740 con 0x7fc6200cfff0 2024-04-25T19:02:56.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.588+0000 7fc61cc00640 1 -- 172.21.15.123:0/4273872014 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc61005fc80 con 0x7fc6200cfff0 2024-04-25T19:02:56.591 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 msgr2=0x7fc5fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 0x7fc5fc0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6080012e0 tx=0x7fc608007d50 comp rx=0 tx=0).stop 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 msgr2=0x7fc6200b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc61000d600 tx=0x7fc61000dac0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 shutdown_connections 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc5fc0744b0 0x7fc5fc0769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620139570 0x7fc62013af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc620143040 0x7fc620139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 --2- 172.21.15.123:0/4273872014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6200cfff0 0x7fc6200b4e60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 >> 172.21.15.123:0/4273872014 conn(0x7fc620073090 msgr2=0x7fc62006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 shutdown_connections 2024-04-25T19:02:56.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.591+0000 7fc625800640 1 -- 172.21.15.123:0/4273872014 wait complete. 2024-04-25T19:02:56.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:02:56.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.877+0000 7fa99be00640 1 Processor -- start 2024-04-25T19:02:56.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.877+0000 7fa99be00640 1 -- start start 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa99c160960 0x7fa99c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99be00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c0d0d50 con 0x7fa99c06d580 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c0d0fd0 con 0x7fa99c160960 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c0d12b0 con 0x7fa99c06d990 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa99c160960 0x7fa99c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa99c160960 0x7fa99c0d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43224/0 (socket says 172.21.15.123:43224) 2024-04-25T19:02:56.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.879+0000 7fa99aa00640 1 -- 172.21.15.123:0/943280562 learned_addr learned my addr 172.21.15.123:0/943280562 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:56.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.880+0000 7fa99b400640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.880+0000 7fa99aa00640 1 -- 172.21.15.123:0/943280562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 msgr2=0x7fa99c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:56.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.880+0000 7fa99aa00640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.880+0000 7fa99aa00640 1 -- 172.21.15.123:0/943280562 >> v1:172.21.15.146:6789/0 conn(0x7fa99c06d580 legacy=0x7fa99c15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:56.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.880+0000 7fa99aa00640 1 -- 172.21.15.123:0/943280562 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c06f130 con 0x7fa99c160960 2024-04-25T19:02:56.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.881+0000 7fa99aa00640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c0d0810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009870 tx=0x7fa98c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b0c2732299b93a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.881+0000 7fa99a000640 1 -- 172.21.15.123:0/943280562 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c037070 con 0x7fa99c160960 2024-04-25T19:02:56.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.881+0000 7fa99a000640 1 -- 172.21.15.123:0/943280562 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c032440 con 0x7fa99c160960 2024-04-25T19:02:56.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.881+0000 7fa99a000640 1 -- 172.21.15.123:0/943280562 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c032810 con 0x7fa99c160960 2024-04-25T19:02:56.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 -- 172.21.15.123:0/943280562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 msgr2=0x7fa99c0d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c0d0810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009870 tx=0x7fa98c02a180 comp rx=0 tx=0).stop 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 -- 172.21.15.123:0/943280562 shutdown_connections 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 --2- 172.21.15.123:0/943280562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c0d0810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 -- 172.21.15.123:0/943280562 >> 172.21.15.123:0/943280562 conn(0x7fa99c073200 msgr2=0x7fa99c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 -- 172.21.15.123:0/943280562 shutdown_connections 2024-04-25T19:02:56.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.882+0000 7fa99be00640 1 -- 172.21.15.123:0/943280562 wait complete. 2024-04-25T19:02:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.885+0000 7fa99be00640 1 Processor -- start 2024-04-25T19:02:56.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.885+0000 7fa99be00640 1 -- start start 2024-04-25T19:02:56.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa99c06d580 0x7fa99c1f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c15cb60 con 0x7fa99c160960 2024-04-25T19:02:56.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c06f6f0 con 0x7fa99c06d580 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.886+0000 7fa99be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c160d70 con 0x7fa99c06d990 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43228/0 (socket says 172.21.15.123:43228) 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 -- 172.21.15.123:0/2560434892 learned_addr learned my addr 172.21.15.123:0/2560434892 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99aa00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa99c06d580 0x7fa99c1f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa993e00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 -- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 msgr2=0x7fa99c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 -- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa99c06d580 msgr2=0x7fa99c1f9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa99c06d580 0x7fa99c1f9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.887+0000 7fa99b400640 1 -- 172.21.15.123:0/2560434892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c1fa4e0 con 0x7fa99c160960 2024-04-25T19:02:56.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa99b400640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa99400cc00 tx=0x7fa99400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99400ce10 con 0x7fa99c160960 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa994003990 con 0x7fa99c160960 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994003d60 con 0x7fa99c160960 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c156550 con 0x7fa99c160960 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.888+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c156d30 con 0x7fa99c160960 2024-04-25T19:02:56.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.889+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c1567a0 con 0x7fa99c160960 2024-04-25T19:02:56.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.891+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa994003f00 con 0x7fa99c160960 2024-04-25T19:02:56.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.892+0000 7fa992a00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 0x7fa9780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.892+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa994095aa0 con 0x7fa99c160960 2024-04-25T19:02:56.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.892+0000 7fa99aa00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 0x7fa9780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.893+0000 7fa99aa00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 0x7fa9780769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c034000 tx=0x7fa98c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:56.897+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa99c1fa4e0 con 0x7fa99c160960 2024-04-25T19:02:57.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.096+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa99c050740 con 0x7fa99c160960 2024-04-25T19:02:57.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.097+0000 7fa992a00640 1 -- 172.21.15.123:0/2560434892 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa994014030 con 0x7fa99c160960 2024-04-25T19:02:57.099 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:02:57.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 msgr2=0x7fa9780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 0x7fa9780769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c034000 tx=0x7fa98c02e000 comp rx=0 tx=0).stop 2024-04-25T19:02:57.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 msgr2=0x7fa99c15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa99400cc00 tx=0x7fa99400a5c0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 shutdown_connections 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa9780744b0 0x7fa9780769c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa99c160960 0x7fa99c15ae50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa99c06d990 0x7fa99c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 --2- 172.21.15.123:0/2560434892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa99c06d580 0x7fa99c1f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.099+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 >> 172.21.15.123:0/2560434892 conn(0x7fa99c073200 msgr2=0x7fa99c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:57.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.100+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 shutdown_connections 2024-04-25T19:02:57.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.100+0000 7fa99be00640 1 -- 172.21.15.123:0/2560434892 wait complete. 2024-04-25T19:02:57.124 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:02:57.124 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:02:57.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:02:57.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.444+0000 7f2ac7200640 1 Processor -- start 2024-04-25T19:02:57.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.444+0000 7f2ac7200640 1 -- start start 2024-04-25T19:02:57.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac7200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac7200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac80cf1d0 con 0x7f2ac8160960 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac7200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac80d0fd0 con 0x7f2ac815c450 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac80d12b0 con 0x7f2ac80ce880 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2abfe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2abfe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:46164/0 (socket says 172.21.15.123:46164) 2024-04-25T19:02:57.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.447+0000 7f2abfe00640 1 -- 172.21.15.123:0/1726025057 learned_addr learned my addr 172.21.15.123:0/1726025057 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:57.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.448+0000 7f2ac6800640 1 -- 172.21.15.123:0/1726025057 >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 msgr2=0x7f2ac815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:57.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.448+0000 7f2ac6800640 1 --2- 172.21.15.123:0/1726025057 >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.448+0000 7f2ac6800640 1 -- 172.21.15.123:0/1726025057 >> v1:172.21.15.146:6789/0 conn(0x7f2ac8160960 legacy=0x7f2ac80d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.448+0000 7f2ac6800640 1 -- 172.21.15.123:0/1726025057 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac806f130 con 0x7f2ac80ce880 2024-04-25T19:02:57.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.448+0000 7f2abfe00640 1 --2- 172.21.15.123:0/1726025057 >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:57.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.449+0000 7f2ac6800640 1 --2- 172.21.15.123:0/1726025057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac80cec90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000aea0 tx=0x7f2ac002b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427fb47a1245e413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.450+0000 7f2ac5400640 1 -- 172.21.15.123:0/1726025057 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ac80ce880 2024-04-25T19:02:57.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.450+0000 7f2ac5400640 1 -- 172.21.15.123:0/1726025057 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0008640 con 0x7f2ac80ce880 2024-04-25T19:02:57.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.450+0000 7f2ac5400640 1 -- 172.21.15.123:0/1726025057 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0008a10 con 0x7f2ac80ce880 2024-04-25T19:02:57.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 -- 172.21.15.123:0/1726025057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 msgr2=0x7f2ac80cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 --2- 172.21.15.123:0/1726025057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac80cec90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000aea0 tx=0x7f2ac002b720 comp rx=0 tx=0).stop 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 -- 172.21.15.123:0/1726025057 shutdown_connections 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 --2- 172.21.15.123:0/1726025057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac80cec90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 --2- 172.21.15.123:0/1726025057 >> v2:172.21.15.146:3300/0 conn(0x7f2ac815c450 0x7f2ac815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 -- 172.21.15.123:0/1726025057 >> 172.21.15.123:0/1726025057 conn(0x7f2ac8073200 msgr2=0x7f2ac8073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 -- 172.21.15.123:0/1726025057 shutdown_connections 2024-04-25T19:02:57.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.451+0000 7f2ac7200640 1 -- 172.21.15.123:0/1726025057 wait complete. 2024-04-25T19:02:57.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.454+0000 7f2ac7200640 1 Processor -- start 2024-04-25T19:02:57.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.454+0000 7f2ac7200640 1 -- start start 2024-04-25T19:02:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac814af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac8160960 0x7f2ac815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8160d70 con 0x7f2ac815c450 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac806f6f0 con 0x7f2ac8160960 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2ac7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac806d580 con 0x7f2ac80ce880 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43238/0 (socket says 172.21.15.123:43238) 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.456+0000 7f2abfe00640 1 -- 172.21.15.123:0/4245802295 learned_addr learned my addr 172.21.15.123:0/4245802295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2ac5e00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac814af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2ac6800640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac8160960 0x7f2ac815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 -- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 msgr2=0x7f2ac814af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac814af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 -- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac8160960 msgr2=0x7f2ac815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac8160960 0x7f2ac815d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 -- 172.21.15.123:0/4245802295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac81f6600 con 0x7f2ac815c450 2024-04-25T19:02:57.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.457+0000 7f2abfe00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9b0 tx=0x7f2ab000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.458+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0017070 con 0x7f2ac815c450 2024-04-25T19:02:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.458+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab000c9a0 con 0x7f2ac815c450 2024-04-25T19:02:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.458+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000cd50 con 0x7f2ac815c450 2024-04-25T19:02:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.458+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac81f68e0 con 0x7f2ac815c450 2024-04-25T19:02:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.458+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac81f6300 con 0x7f2ac815c450 2024-04-25T19:02:57.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.459+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac8050740 con 0x7f2ac815c450 2024-04-25T19:02:57.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.460+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ab001a070 con 0x7f2ac815c450 2024-04-25T19:02:57.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.461+0000 7f2abe000640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 0x7f2aa40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.461+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2ab0095740 con 0x7f2ac815c450 2024-04-25T19:02:57.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.461+0000 7f2ac5e00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 0x7f2aa40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.462+0000 7f2ac5e00640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 0x7f2aa40769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8000f60 tx=0x7f2ab8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.466+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ab005fc10 con 0x7f2ac815c450 2024-04-25T19:02:57.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.671+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ac8154380 con 0x7f2ac815c450 2024-04-25T19:02:57.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.674+0000 7f2abe000640 1 -- 172.21.15.123:0/4245802295 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f2ab005f510 con 0x7f2ac815c450 2024-04-25T19:02:57.677 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:02:57.677 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.677+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 msgr2=0x7f2aa40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.677+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 0x7f2aa40769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8000f60 tx=0x7f2ab8009040 comp rx=0 tx=0).stop 2024-04-25T19:02:57.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.677+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 msgr2=0x7f2ac81538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.677+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9b0 tx=0x7f2ab000ee70 comp rx=0 tx=0).stop 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 shutdown_connections 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2aa40744e0 0x7f2aa40769f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac8160960 0x7f2ac815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac815c450 0x7f2ac81538c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 --2- 172.21.15.123:0/4245802295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80ce880 0x7f2ac814af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 >> 172.21.15.123:0/4245802295 conn(0x7f2ac8073200 msgr2=0x7f2ac80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 shutdown_connections 2024-04-25T19:02:57.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.678+0000 7f2ac7200640 1 -- 172.21.15.123:0/4245802295 wait complete. 2024-04-25T19:02:57.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:02:57.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:02:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.977+0000 7fb81ca00640 1 Processor -- start 2024-04-25T19:02:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.978+0000 7fb81ca00640 1 -- start start 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.979+0000 7fb81ca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb81806d580 0x7fb81806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.979+0000 7fb81ca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81806ded0 con 0x7fb81814da70 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb81ca00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8181653e0 con 0x7fb8180b4f40 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb81ca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb8180b44f0 con 0x7fb81806d580 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb80de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb81806d580 0x7fb81806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb816800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb81814da70 0x7fb818164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47986/0 (socket says 172.21.15.123:47986) 2024-04-25T19:02:57.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb816800640 1 -- 172.21.15.123:0/4141008268 learned_addr learned my addr 172.21.15.123:0/4141008268 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb80de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb81806d580 0x7fb81806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43254/0 (socket says 172.21.15.123:43254) 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb80de00640 1 -- 172.21.15.123:0/4141008268 >> v1:172.21.15.146:6789/0 conn(0x7fb8180b4f40 legacy=0x7fb8180b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb80de00640 1 -- 172.21.15.123:0/4141008268 >> v1:172.21.15.123:6789/0 conn(0x7fb81814da70 legacy=0x7fb818164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb80de00640 1 -- 172.21.15.123:0/4141008268 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181656c0 con 0x7fb81806d580 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.980+0000 7fb815e00640 1 -- 172.21.15.123:0/4141008268 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757803964 0 0) 0x7fb808001110 con 0x7fb81814da70 2024-04-25T19:02:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.981+0000 7fb80de00640 1 --2- 172.21.15.123:0/4141008268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb81806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8000097a0 tx=0x7fb80002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c043f854dbbc5ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.982+0000 7fb815e00640 1 -- 172.21.15.123:0/4141008268 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800037070 con 0x7fb81806d580 2024-04-25T19:02:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.982+0000 7fb815e00640 1 -- 172.21.15.123:0/4141008268 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80002ae50 con 0x7fb81806d580 2024-04-25T19:02:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.982+0000 7fb815e00640 1 -- 172.21.15.123:0/4141008268 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800032660 con 0x7fb81806d580 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.982+0000 7fb81ca00640 1 -- 172.21.15.123:0/4141008268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 msgr2=0x7fb81806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.982+0000 7fb81ca00640 1 --2- 172.21.15.123:0/4141008268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb81806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8000097a0 tx=0x7fb80002a140 comp rx=0 tx=0).stop 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.983+0000 7fb81ca00640 1 -- 172.21.15.123:0/4141008268 shutdown_connections 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.983+0000 7fb81ca00640 1 --2- 172.21.15.123:0/4141008268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb81806d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.983+0000 7fb81ca00640 1 -- 172.21.15.123:0/4141008268 >> 172.21.15.123:0/4141008268 conn(0x7fb818073090 msgr2=0x7fb8180734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.983+0000 7fb81ca00640 1 -- 172.21.15.123:0/4141008268 shutdown_connections 2024-04-25T19:02:57.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.983+0000 7fb81ca00640 1 -- 172.21.15.123:0/4141008268 wait complete. 2024-04-25T19:02:57.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.985+0000 7fb81ca00640 1 Processor -- start 2024-04-25T19:02:57.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.985+0000 7fb81ca00640 1 -- start start 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8180b4f40 0x7fb818146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 0x7fb818160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb81814de80 con 0x7fb81806d580 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180b4a20 con 0x7fb81814da70 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb81ca00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb818165030 con 0x7fb8180b4f40 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43258/0 (socket says 172.21.15.123:43258) 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 -- 172.21.15.123:0/2765144058 learned_addr learned my addr 172.21.15.123:0/2765144058 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb80de00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8180b4f40 0x7fb818146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb817200640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 0x7fb818160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 -- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8180b4f40 msgr2=0x7fb818146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8180b4f40 0x7fb818146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 -- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 msgr2=0x7fb818160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 0x7fb818160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb816800640 1 -- 172.21.15.123:0/2765144058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb818166280 con 0x7fb81806d580 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.987+0000 7fb817200640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 0x7fb818160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb816800640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c410 tx=0x7fb80800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8080074e0 con 0x7fb81806d580 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8080047e0 con 0x7fb81806d580 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb818166560 con 0x7fb81806d580 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808004bd0 con 0x7fb81806d580 2024-04-25T19:02:57.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.988+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb818166b20 con 0x7fb81806d580 2024-04-25T19:02:57.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.989+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818050740 con 0x7fb81806d580 2024-04-25T19:02:57.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.991+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb808011070 con 0x7fb81806d580 2024-04-25T19:02:57.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.992+0000 7fb80f400640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 0x7fb7f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.993+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb808095180 con 0x7fb81806d580 2024-04-25T19:02:57.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.993+0000 7fb80de00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 0x7fb7f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.993+0000 7fb80de00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 0x7fb7f40769c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb800034000 tx=0x7fb80002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:57.998+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb81806ded0 con 0x7fb81806d580 2024-04-25T19:02:58.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.198+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8180742b0 con 0x7fb81806d580 2024-04-25T19:02:58.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.199+0000 7fb80f400640 1 -- 172.21.15.123:0/2765144058 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb80805f660 con 0x7fb81806d580 2024-04-25T19:02:58.201 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 msgr2=0x7fb7f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 0x7fb7f40769c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb800034000 tx=0x7fb80002e000 comp rx=0 tx=0).stop 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 msgr2=0x7fb818165a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c410 tx=0x7fb80800c8d0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 shutdown_connections 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7f40744b0 0x7fb7f40769c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb81814da70 0x7fb818160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8180b4f40 0x7fb818146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 --2- 172.21.15.123:0/2765144058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb81806d580 0x7fb818165a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 >> 172.21.15.123:0/2765144058 conn(0x7fb818073090 msgr2=0x7fb81814cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 shutdown_connections 2024-04-25T19:02:58.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.202+0000 7fb81ca00640 1 -- 172.21.15.123:0/2765144058 wait complete. 2024-04-25T19:02:58.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:02:58.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.499+0000 7f6055c00640 1 Processor -- start 2024-04-25T19:02:58.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.500+0000 7f6055c00640 1 -- start start 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6055c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f60500cd140 0x7f605015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6055c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 0x7f60500af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6055c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60500cc5f0 con 0x7f605006daa0 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6055c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f605006deb0 con 0x7f60500cd140 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6055c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f605006d610 con 0x7f6050162540 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f604ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f605006daa0 0x7f60500cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47988/0 (socket says 172.21.15.123:47988) 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f604ea00640 1 -- 172.21.15.123:0/1259425306 learned_addr learned my addr 172.21.15.123:0/1259425306 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f6046000640 1 --2- 172.21.15.123:0/1259425306 >> v2:172.21.15.123:3300/0 conn(0x7f60500cd140 0x7f605015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.502+0000 7f604f400640 1 --2- 172.21.15.123:0/1259425306 >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 0x7f60500af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f604e000640 1 -- 172.21.15.123:0/1259425306 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512452826 0 0) 0x7f60500cc5f0 con 0x7f605006daa0 2024-04-25T19:02:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f604e000640 1 -- 172.21.15.123:0/1259425306 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6034003520 con 0x7f605006daa0 2024-04-25T19:02:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f6046000640 1 -- 172.21.15.123:0/1259425306 >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 msgr2=0x7f60500af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f6046000640 1 --2- 172.21.15.123:0/1259425306 >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 0x7f60500af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f6046000640 1 -- 172.21.15.123:0/1259425306 >> v1:172.21.15.123:6789/0 conn(0x7f605006daa0 legacy=0x7f60500cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f6046000640 1 -- 172.21.15.123:0/1259425306 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605006ca00 con 0x7f60500cd140 2024-04-25T19:02:58.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.503+0000 7f604f400640 1 --2- 172.21.15.123:0/1259425306 >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 0x7f60500af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.504+0000 7f6046000640 1 --2- 172.21.15.123:0/1259425306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60500cd140 0x7f605015dfc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60500d01c0 tx=0x7f603802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8db335d7aa97284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.504+0000 7f604e000640 1 -- 172.21.15.123:0/1259425306 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038037070 con 0x7f60500cd140 2024-04-25T19:02:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.504+0000 7f604e000640 1 -- 172.21.15.123:0/1259425306 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6038032440 con 0x7f60500cd140 2024-04-25T19:02:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.504+0000 7f604e000640 1 -- 172.21.15.123:0/1259425306 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038032810 con 0x7f60500cd140 2024-04-25T19:02:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 -- 172.21.15.123:0/1259425306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60500cd140 msgr2=0x7f605015dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 --2- 172.21.15.123:0/1259425306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60500cd140 0x7f605015dfc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60500d01c0 tx=0x7f603802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 -- 172.21.15.123:0/1259425306 shutdown_connections 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 --2- 172.21.15.123:0/1259425306 >> v2:172.21.15.146:3300/0 conn(0x7f6050162540 0x7f60500af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 --2- 172.21.15.123:0/1259425306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60500cd140 0x7f605015dfc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 -- 172.21.15.123:0/1259425306 >> 172.21.15.123:0/1259425306 conn(0x7f60500ce390 msgr2=0x7f60500ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 -- 172.21.15.123:0/1259425306 shutdown_connections 2024-04-25T19:02:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.505+0000 7f6055c00640 1 -- 172.21.15.123:0/1259425306 wait complete. 2024-04-25T19:02:58.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.508+0000 7f6055c00640 1 Processor -- start 2024-04-25T19:02:58.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.508+0000 7f6055c00640 1 -- start start 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 0x7f605015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6050162540 0x7f60500b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f605006d2a0 con 0x7f605014f650 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f605006cfc0 con 0x7f6050162540 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.510+0000 7f6055c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6050162950 con 0x7f605006daa0 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43272/0 (socket says 172.21.15.123:43272) 2024-04-25T19:02:58.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 -- 172.21.15.123:0/1785570515 learned_addr learned my addr 172.21.15.123:0/1785570515 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604ea00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 0x7f605015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f6046000640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6050162540 0x7f60500b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 -- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 msgr2=0x7f605015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 0x7f605015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 -- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6050162540 msgr2=0x7f60500b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6050162540 0x7f60500b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604f400640 1 -- 172.21.15.123:0/1785570515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6050159d00 con 0x7f605014f650 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.511+0000 7f604ea00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 0x7f605015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:58.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f604f400640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f604800d5e0 tx=0x7f604800daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048018070 con 0x7f605014f650 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6048004030 con 0x7f605014f650 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6050159fe0 con 0x7f605014f650 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048004400 con 0x7f605014f650 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.512+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6050153030 con 0x7f605014f650 2024-04-25T19:02:58.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.513+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605014fa60 con 0x7f605014f650 2024-04-25T19:02:58.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.514+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6048014620 con 0x7f605014f650 2024-04-25T19:02:58.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.515+0000 7f6047400640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 0x7f602c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.515+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f60480957d0 con 0x7f605014f650 2024-04-25T19:02:58.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.515+0000 7f604ea00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 0x7f602c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.516+0000 7f604ea00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 0x7f602c0769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6040005c00 tx=0x7f6040009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.520+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f605014fa60 con 0x7f605014f650 2024-04-25T19:02:58.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.717+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6050076a40 con 0x7f605014f650 2024-04-25T19:02:58.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.718+0000 7f6047400640 1 -- 172.21.15.123:0/1785570515 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f604805fcb0 con 0x7f605014f650 2024-04-25T19:02:58.720 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:02:58.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.720+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 msgr2=0x7f602c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.720+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 0x7f602c0769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6040005c00 tx=0x7f6040009040 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.720+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 msgr2=0x7f60500b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.720+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f604800d5e0 tx=0x7f604800daa0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 shutdown_connections 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f602c0744b0 0x7f602c0769c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605014f650 0x7f60500b6c00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6050162540 0x7f60500b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 --2- 172.21.15.123:0/1785570515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f605006daa0 0x7f605015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 >> 172.21.15.123:0/1785570515 conn(0x7f60500ce390 msgr2=0x7f60500b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 shutdown_connections 2024-04-25T19:02:58.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:58.721+0000 7f6055c00640 1 -- 172.21.15.123:0/1785570515 wait complete. 2024-04-25T19:02:58.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:02:59.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.007+0000 7fef58a00640 1 Processor -- start 2024-04-25T19:02:59.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.007+0000 7fef58a00640 1 -- start start 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.009+0000 7fef58a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 0x7fef540b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.009+0000 7fef58a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef540b4390 0x7fef540b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.009+0000 7fef58a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef540b1a30 con 0x7fef540cf8b0 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.009+0000 7fef58a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fef540b2040 con 0x7fef540b4390 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.009+0000 7fef58a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fef540b4d50 con 0x7fef540b3a40 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef540b4390 0x7fef540b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef540b4390 0x7fef540b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43274/0 (socket says 172.21.15.123:43274) 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 -- 172.21.15.123:0/1991433772 learned_addr learned my addr 172.21.15.123:0/1991433772 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:59.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef51400640 1 --2- 172.21.15.123:0/1991433772 >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 0x7fef540b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 -- 172.21.15.123:0/1991433772 >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 msgr2=0x7fef540b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:59.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 --2- 172.21.15.123:0/1991433772 >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 0x7fef540b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 -- 172.21.15.123:0/1991433772 >> v1:172.21.15.146:6789/0 conn(0x7fef540cf8b0 legacy=0x7fef54160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:59.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef52800640 1 -- 172.21.15.123:0/1991433772 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef540b5030 con 0x7fef540b4390 2024-04-25T19:02:59.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.010+0000 7fef51400640 1 --2- 172.21.15.123:0/1991433772 >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 0x7fef540b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:59.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.011+0000 7fef52800640 1 --2- 172.21.15.123:0/1991433772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b4390 0x7fef540b24d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef540d3860 tx=0x7fef4c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4eae3951e61766 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.011+0000 7fef50a00640 1 -- 172.21.15.123:0/1991433772 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c036070 con 0x7fef540b4390 2024-04-25T19:02:59.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.011+0000 7fef50a00640 1 -- 172.21.15.123:0/1991433772 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c031600 con 0x7fef540b4390 2024-04-25T19:02:59.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.011+0000 7fef50a00640 1 -- 172.21.15.123:0/1991433772 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0319d0 con 0x7fef540b4390 2024-04-25T19:02:59.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 -- 172.21.15.123:0/1991433772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b4390 msgr2=0x7fef540b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 --2- 172.21.15.123:0/1991433772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b4390 0x7fef540b24d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef540d3860 tx=0x7fef4c02a660 comp rx=0 tx=0).stop 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 -- 172.21.15.123:0/1991433772 shutdown_connections 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 --2- 172.21.15.123:0/1991433772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b4390 0x7fef540b24d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 --2- 172.21.15.123:0/1991433772 >> v2:172.21.15.146:3300/0 conn(0x7fef540b3a40 0x7fef540b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 -- 172.21.15.123:0/1991433772 >> 172.21.15.123:0/1991433772 conn(0x7fef5409b4a0 msgr2=0x7fef5409b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 -- 172.21.15.123:0/1991433772 shutdown_connections 2024-04-25T19:02:59.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.012+0000 7fef58a00640 1 -- 172.21.15.123:0/1991433772 wait complete. 2024-04-25T19:02:59.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.014+0000 7fef58a00640 1 Processor -- start 2024-04-25T19:02:59.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.015+0000 7fef58a00640 1 -- start start 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 0x7fef5415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef540cf8b0 0x7fef540b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef540b4920 con 0x7fef540b3a40 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fef540b47a0 con 0x7fef540cf8b0 2024-04-25T19:02:59.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef58a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef540b4aa0 con 0x7fef540b4390 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef51e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef51e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43288/0 (socket says 172.21.15.123:43288) 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.016+0000 7fef51e00640 1 -- 172.21.15.123:0/1310978204 learned_addr learned my addr 172.21.15.123:0/1310978204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef52800640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef540cf8b0 0x7fef540b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51400640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 0x7fef5415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 -- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 msgr2=0x7fef5415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 0x7fef5415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 -- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef540cf8b0 msgr2=0x7fef540b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef540cf8b0 0x7fef540b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 -- 172.21.15.123:0/1310978204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5415d490 con 0x7fef540b3a40 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51400640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 0x7fef5415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef51e00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b3c0 tx=0x7fef4400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44011830 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef44016070 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5415d770 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44011c30 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.017+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5415dd30 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.019+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef54076a40 con 0x7fef540b3a40 2024-04-25T19:02:59.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.020+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef4401b460 con 0x7fef540b3a40 2024-04-25T19:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.021+0000 7fef4a000640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 0x7fef300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.021+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fef44095260 con 0x7fef540b3a40 2024-04-25T19:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.021+0000 7fef51400640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 0x7fef300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.022+0000 7fef51400640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 0x7fef300769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006fd0 tx=0x7fef3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.026+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef4405f6a0 con 0x7fef540b3a40 2024-04-25T19:02:59.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.225+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef541602d0 con 0x7fef540b3a40 2024-04-25T19:02:59.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.226+0000 7fef4a000640 1 -- 172.21.15.123:0/1310978204 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fef4405efa0 con 0x7fef540b3a40 2024-04-25T19:02:59.228 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:02:59.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 msgr2=0x7fef300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 0x7fef300769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006fd0 tx=0x7fef3c008040 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 msgr2=0x7fef541f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b3c0 tx=0x7fef4400b880 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 shutdown_connections 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef300744e0 0x7fef300769f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef540cf8b0 0x7fef540b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef540b4390 0x7fef5415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 --2- 172.21.15.123:0/1310978204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef540b3a40 0x7fef541f5fa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.228+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 >> 172.21.15.123:0/1310978204 conn(0x7fef5409b4a0 msgr2=0x7fef540d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.229+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 shutdown_connections 2024-04-25T19:02:59.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.229+0000 7fef58a00640 1 -- 172.21.15.123:0/1310978204 wait complete. 2024-04-25T19:02:59.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:02:59.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.533+0000 7fd061400640 1 Processor -- start 2024-04-25T19:02:59.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.533+0000 7fd061400640 1 -- start start 2024-04-25T19:02:59.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.535+0000 7fd061400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.535+0000 7fd061400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 0x7fd05c06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.535+0000 7fd061400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05c0d5510 con 0x7fd05c0d44d0 2024-04-25T19:02:59.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.535+0000 7fd061400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd05c15e0a0 con 0x7fd05c15dc90 2024-04-25T19:02:59.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.535+0000 7fd061400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c0d11c0 con 0x7fd05c0d0db0 2024-04-25T19:02:59.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.536+0000 7fd05b400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.536+0000 7fd060a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 0x7fd05c06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.536+0000 7fd05b400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:46214/0 (socket says 172.21.15.123:46214) 2024-04-25T19:02:59.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.536+0000 7fd05b400640 1 -- 172.21.15.123:0/1818599977 learned_addr learned my addr 172.21.15.123:0/1818599977 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:59.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.537+0000 7fd060a00640 1 -- 172.21.15.123:0/1818599977 >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 msgr2=0x7fd05c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:59.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.537+0000 7fd060a00640 1 --2- 172.21.15.123:0/1818599977 >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.537+0000 7fd060a00640 1 -- 172.21.15.123:0/1818599977 >> v1:172.21.15.146:6789/0 conn(0x7fd05c0d44d0 legacy=0x7fd05c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.537+0000 7fd060a00640 1 -- 172.21.15.123:0/1818599977 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0d4040 con 0x7fd05c0d0db0 2024-04-25T19:02:59.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.537+0000 7fd05b400640 1 --2- 172.21.15.123:0/1818599977 >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:59.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.538+0000 7fd060a00640 1 --2- 172.21.15.123:0/1818599977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 0x7fd05c06c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0540097a0 tx=0x7fd05402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dacb2273316787b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.539+0000 7fd05aa00640 1 -- 172.21.15.123:0/1818599977 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037070 con 0x7fd05c0d0db0 2024-04-25T19:02:59.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.539+0000 7fd05aa00640 1 -- 172.21.15.123:0/1818599977 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05402ae50 con 0x7fd05c0d0db0 2024-04-25T19:02:59.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.539+0000 7fd05aa00640 1 -- 172.21.15.123:0/1818599977 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054032660 con 0x7fd05c0d0db0 2024-04-25T19:02:59.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.539+0000 7fd061400640 1 -- 172.21.15.123:0/1818599977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 msgr2=0x7fd05c06c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.539+0000 7fd061400640 1 --2- 172.21.15.123:0/1818599977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 0x7fd05c06c8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0540097a0 tx=0x7fd05402a100 comp rx=0 tx=0).stop 2024-04-25T19:02:59.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 -- 172.21.15.123:0/1818599977 shutdown_connections 2024-04-25T19:02:59.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 --2- 172.21.15.123:0/1818599977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d0db0 0x7fd05c06c8c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 --2- 172.21.15.123:0/1818599977 >> v2:172.21.15.146:3300/0 conn(0x7fd05c15dc90 0x7fd05c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 -- 172.21.15.123:0/1818599977 >> 172.21.15.123:0/1818599977 conn(0x7fd05c0d2540 msgr2=0x7fd05c0d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 -- 172.21.15.123:0/1818599977 shutdown_connections 2024-04-25T19:02:59.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.540+0000 7fd061400640 1 -- 172.21.15.123:0/1818599977 wait complete. 2024-04-25T19:02:59.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.542+0000 7fd061400640 1 Processor -- start 2024-04-25T19:02:59.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.542+0000 7fd061400640 1 -- start start 2024-04-25T19:02:59.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.543+0000 7fd061400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0d0db0 0x7fd05c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.543+0000 7fd061400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c15dc90 0x7fd05c0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.543+0000 7fd061400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd061400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0d4f50 con 0x7fd05c0ca040 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd061400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0d4c70 con 0x7fd05c0d0db0 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd061400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c0d5230 con 0x7fd05c15dc90 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43304/0 (socket says 172.21.15.123:43304) 2024-04-25T19:02:59.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 -- 172.21.15.123:0/640270215 learned_addr learned my addr 172.21.15.123:0/640270215 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:02:59.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd05b400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c15dc90 0x7fd05c0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd05be00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0d0db0 0x7fd05c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 -- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c15dc90 msgr2=0x7fd05c0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c15dc90 0x7fd05c0c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 -- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0d0db0 msgr2=0x7fd05c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0d0db0 0x7fd05c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.544+0000 7fd060a00640 1 -- 172.21.15.123:0/640270215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0c90e0 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd060a00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd05402a140 tx=0x7fd0540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054041030 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05402ac50 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037040 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c0cb730 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.545+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c0cbcf0 con 0x7fd05c0ca040 2024-04-25T19:02:59.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.546+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c050740 con 0x7fd05c0ca040 2024-04-25T19:02:59.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.547+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd054044070 con 0x7fd05c0ca040 2024-04-25T19:02:59.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.549+0000 7fd053e00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 0x7fd0380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.549+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd0540b5aa0 con 0x7fd05c0ca040 2024-04-25T19:02:59.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.549+0000 7fd05be00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 0x7fd0380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.550+0000 7fd05be00640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 0x7fd0380769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0d3be0 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.554+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd05c0c90e0 con 0x7fd05c0ca040 2024-04-25T19:02:59.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.752+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd05c0cb4c0 con 0x7fd05c0ca040 2024-04-25T19:02:59.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.753+0000 7fd053e00640 1 -- 172.21.15.123:0/640270215 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd05402ae20 con 0x7fd05c0ca040 2024-04-25T19:02:59.755 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:02:59.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 msgr2=0x7fd0380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 0x7fd0380769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0d3be0 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-04-25T19:02:59.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 msgr2=0x7fd05c15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd05402a140 tx=0x7fd0540046c0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 shutdown_connections 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd0380744b0 0x7fd0380769c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd05c0ca040 0x7fd05c15a030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c15dc90 0x7fd05c0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.755+0000 7fd061400640 1 --2- 172.21.15.123:0/640270215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0d0db0 0x7fd05c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.756+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 >> 172.21.15.123:0/640270215 conn(0x7fd05c0d2540 msgr2=0x7fd05c15ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.756+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 shutdown_connections 2024-04-25T19:02:59.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:02:59.756+0000 7fd061400640 1 -- 172.21.15.123:0/640270215 wait complete. 2024-04-25T19:02:59.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:03:00.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.036+0000 7fcc0f000640 1 Processor -- start 2024-04-25T19:03:00.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.036+0000 7fcc0f000640 1 -- start start 2024-04-25T19:03:00.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.038+0000 7fcc0f000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc0806fe20 0x7fcc0815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0f000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0815f070 con 0x7fcc0806d580 2024-04-25T19:03:00.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0f000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0806f340 con 0x7fcc0815e2f0 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0f000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0815ee10 con 0x7fcc0806fe20 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0dc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc0806fe20 0x7fcc0815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0d200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcc0806d580 0x7fcc0806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47990/0 (socket says 172.21.15.123:47990) 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0dc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc0806fe20 0x7fcc0815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43316/0 (socket says 172.21.15.123:43316) 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0d200640 1 -- 172.21.15.123:0/3626488306 learned_addr learned my addr 172.21.15.123:0/3626488306 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0dc00640 1 -- 172.21.15.123:0/3626488306 >> v1:172.21.15.146:6789/0 conn(0x7fcc0815e2f0 legacy=0x7fcc0815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0dc00640 1 -- 172.21.15.123:0/3626488306 >> v1:172.21.15.123:6789/0 conn(0x7fcc0806d580 legacy=0x7fcc0806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc0dc00640 1 -- 172.21.15.123:0/3626488306 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0806f620 con 0x7fcc0806fe20 2024-04-25T19:03:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.039+0000 7fcc07e00640 1 -- 172.21.15.123:0/3626488306 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419292966 0 0) 0x7fcc0815f070 con 0x7fcc0806d580 2024-04-25T19:03:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.040+0000 7fcc0dc00640 1 --2- 172.21.15.123:0/3626488306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0806fe20 0x7fcc0815b6b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eab96950ba91d5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc07e00640 1 -- 172.21.15.123:0/3626488306 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8037070 con 0x7fcc0806fe20 2024-04-25T19:03:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc07e00640 1 -- 172.21.15.123:0/3626488306 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf802ae50 con 0x7fcc0806fe20 2024-04-25T19:03:00.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc07e00640 1 -- 172.21.15.123:0/3626488306 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8032660 con 0x7fcc0806fe20 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 -- 172.21.15.123:0/3626488306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0806fe20 msgr2=0x7fcc0815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3626488306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0806fe20 0x7fcc0815b6b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802a140 comp rx=0 tx=0).stop 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 -- 172.21.15.123:0/3626488306 shutdown_connections 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3626488306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0806fe20 0x7fcc0815b6b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 -- 172.21.15.123:0/3626488306 >> 172.21.15.123:0/3626488306 conn(0x7fcc08073200 msgr2=0x7fcc08073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 -- 172.21.15.123:0/3626488306 shutdown_connections 2024-04-25T19:03:00.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.041+0000 7fcc0f000640 1 -- 172.21.15.123:0/3626488306 wait complete. 2024-04-25T19:03:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.044+0000 7fcc0f000640 1 Processor -- start 2024-04-25T19:03:00.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.044+0000 7fcc0f000640 1 -- start start 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc0806d580 0x7fcc08166e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc0806fe20 0x7fcc08149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08168c70 con 0x7fcc0815e2f0 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0815bbf0 con 0x7fcc0806d580 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0f000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcc08168df0 con 0x7fcc0806fe20 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.045+0000 7fcc0e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43320/0 (socket says 172.21.15.123:43320) 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 -- 172.21.15.123:0/3393006352 learned_addr learned my addr 172.21.15.123:0/3393006352 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0dc00640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc0806d580 0x7fcc08166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0d200640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc0806fe20 0x7fcc08149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 -- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc0806fe20 msgr2=0x7fcc08149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc0806fe20 0x7fcc08149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 -- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc0806d580 msgr2=0x7fcc08166e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc0806d580 0x7fcc08166e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 -- 172.21.15.123:0/3393006352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc081686f0 con 0x7fcc0815e2f0 2024-04-25T19:03:00.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc0e600640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000b490 tx=0x7fcc0000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.046+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00015020 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.047+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00011850 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.047+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc08167650 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.047+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011c40 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.047+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc08167c10 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.048+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0815aef0 con 0x7fcc0815e2f0 2024-04-25T19:03:00.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.049+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc00018020 con 0x7fcc0815e2f0 2024-04-25T19:03:00.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.050+0000 7fcc05600640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 0x7fcbe80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.050+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcc00095410 con 0x7fcc0815e2f0 2024-04-25T19:03:00.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.050+0000 7fcc0dc00640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 0x7fcbe80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.051+0000 7fcc0dc00640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 0x7fcbe80769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.055+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc0001f020 con 0x7fcc0815e2f0 2024-04-25T19:03:00.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.274+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc08070a00 con 0x7fcc0815e2f0 2024-04-25T19:03:00.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.275+0000 7fcc05600640 1 -- 172.21.15.123:0/3393006352 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcc0005f860 con 0x7fcc0815e2f0 2024-04-25T19:03:00.276 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:03:00.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.277+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 msgr2=0x7fcbe80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.277+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 0x7fcbe80769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:00.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.277+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 msgr2=0x7fcc081f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.277+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000b490 tx=0x7fcc0000b950 comp rx=0 tx=0).stop 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 shutdown_connections 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcbe80744b0 0x7fcbe80769c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0815e2f0 0x7fcc081f9d70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc0806fe20 0x7fcc08149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 --2- 172.21.15.123:0/3393006352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc0806d580 0x7fcc08166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 >> 172.21.15.123:0/3393006352 conn(0x7fcc08073200 msgr2=0x7fcc0806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 shutdown_connections 2024-04-25T19:03:00.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.278+0000 7fcc0f000640 1 -- 172.21.15.123:0/3393006352 wait complete. 2024-04-25T19:03:00.307 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-25T19:03:00.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:00.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T19:03:00.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40205]: fuse finished with error 0 and tester_r 0 2024-04-25T19:03:00.396 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:03:00.447 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:03:00.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:00.447 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T19:03:00.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:00.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.785+0000 7f3c9fe00640 1 Processor -- start 2024-04-25T19:03:00.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.785+0000 7f3c9fe00640 1 -- start start 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3c9815df00 0x7f3c9806f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9fe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c9806f780 0x7f3c980b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c980b6140 con 0x7f3c980b5d30 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9815e310 con 0x7f3c9815df00 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9fe00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c980b5360 con 0x7f3c9806f780 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3c9815df00 0x7f3c9806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9de00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3c980b5d30 0x7f3c980ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47996/0 (socket says 172.21.15.123:47996) 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.787+0000 7f3c9de00640 1 -- 172.21.15.123:0/1047803376 learned_addr learned my addr 172.21.15.123:0/1047803376 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c9ca00640 1 --2- 172.21.15.123:0/1047803376 >> v2:172.21.15.146:3300/0 conn(0x7f3c9806f780 0x7f3c980b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c9d400640 1 -- 172.21.15.123:0/1047803376 >> v2:172.21.15.146:3300/0 conn(0x7f3c9806f780 msgr2=0x7f3c980b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c9d400640 1 --2- 172.21.15.123:0/1047803376 >> v2:172.21.15.146:3300/0 conn(0x7f3c9806f780 0x7f3c980b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c9d400640 1 -- 172.21.15.123:0/1047803376 >> v1:172.21.15.123:6789/0 conn(0x7f3c980b5d30 legacy=0x7f3c980ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c9d400640 1 -- 172.21.15.123:0/1047803376 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c980b4750 con 0x7f3c9815df00 2024-04-25T19:03:00.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.788+0000 7f3c97e00640 1 -- 172.21.15.123:0/1047803376 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226928211 0 0) 0x7f3c980b6140 con 0x7f3c980b5d30 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.789+0000 7f3c9d400640 1 --2- 172.21.15.123:0/1047803376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9815df00 0x7f3c9806f240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c98072780 tx=0x7f3c8c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b775ff24e9ae7912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.789+0000 7f3c97e00640 1 -- 172.21.15.123:0/1047803376 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c037070 con 0x7f3c9815df00 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.789+0000 7f3c97e00640 1 -- 172.21.15.123:0/1047803376 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c032440 con 0x7f3c9815df00 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.789+0000 7f3c97e00640 1 -- 172.21.15.123:0/1047803376 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02ae60 con 0x7f3c9815df00 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 -- 172.21.15.123:0/1047803376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9815df00 msgr2=0x7f3c9806f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/1047803376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9815df00 0x7f3c9806f240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c98072780 tx=0x7f3c8c02a110 comp rx=0 tx=0).stop 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 -- 172.21.15.123:0/1047803376 shutdown_connections 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/1047803376 >> v2:172.21.15.146:3300/0 conn(0x7f3c9806f780 0x7f3c980b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/1047803376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9815df00 0x7f3c9806f240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 -- 172.21.15.123:0/1047803376 >> 172.21.15.123:0/1047803376 conn(0x7f3c980d1ae0 msgr2=0x7f3c980d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:00.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 -- 172.21.15.123:0/1047803376 shutdown_connections 2024-04-25T19:03:00.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.790+0000 7f3c9fe00640 1 -- 172.21.15.123:0/1047803376 wait complete. 2024-04-25T19:03:00.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.793+0000 7f3c9fe00640 1 Processor -- start 2024-04-25T19:03:00.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.793+0000 7f3c9fe00640 1 -- start start 2024-04-25T19:03:00.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c980b5d30 0x7f3c98151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9815df00 0x7f3c9814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980b4ff0 con 0x7f3c9806f780 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980b4d10 con 0x7f3c980b5d30 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.794+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c980aeef0 con 0x7f3c9815df00 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43340/0 (socket says 172.21.15.123:43340) 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 -- 172.21.15.123:0/2117186119 learned_addr learned my addr 172.21.15.123:0/2117186119 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9ca00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c980b5d30 0x7f3c98151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9de00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9815df00 0x7f3c9814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 -- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9815df00 msgr2=0x7f3c9814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9815df00 0x7f3c9814f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 -- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c980b5d30 msgr2=0x7f3c98151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:00.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c980b5d30 0x7f3c98151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:00.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 -- 172.21.15.123:0/2117186119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c98151ba0 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.795+0000 7f3c9d400640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.796+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c037040 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.796+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c042070 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.796+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c030bd0 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.796+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9814fde0 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.796+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981503a0 con 0x7f3c9806f780 2024-04-25T19:03:00.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.797+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c98076a40 con 0x7f3c9806f780 2024-04-25T19:03:00.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.798+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c8c047460 con 0x7f3c9806f780 2024-04-25T19:03:00.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.799+0000 7f3c95600640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 0x7f3c780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:00.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.799+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3c8c0b6050 con 0x7f3c9806f780 2024-04-25T19:03:00.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.800+0000 7f3c9ca00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 0x7f3c780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:00.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.800+0000 7f3c9ca00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 0x7f3c780769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006fd0 tx=0x7f3c84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:00.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:00.804+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c8c032e20 con 0x7f3c9806f780 2024-04-25T19:03:01.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.004+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c9806f360 con 0x7f3c9806f780 2024-04-25T19:03:01.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.007+0000 7f3c95600640 1 -- 172.21.15.123:0/2117186119 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f3c8c080400 con 0x7f3c9806f780 2024-04-25T19:03:01.009 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:01.009 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:01.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 msgr2=0x7f3c780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 0x7f3c780769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006fd0 tx=0x7f3c84008040 comp rx=0 tx=0).stop 2024-04-25T19:03:01.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 msgr2=0x7f3c98153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 shutdown_connections 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3c780744b0 0x7f3c780769c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9815df00 0x7f3c9814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c980b5d30 0x7f3c98151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 --2- 172.21.15.123:0/2117186119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c9806f780 0x7f3c98153110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 >> 172.21.15.123:0/2117186119 conn(0x7f3c980d1ae0 msgr2=0x7f3c98157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.010+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 shutdown_connections 2024-04-25T19:03:01.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.011+0000 7f3c9fe00640 1 -- 172.21.15.123:0/2117186119 wait complete. 2024-04-25T19:03:01.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.309+0000 7fce4a200640 1 Processor -- start 2024-04-25T19:03:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.309+0000 7fce4a200640 1 -- start start 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.311+0000 7fce4a200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce440b7450 0x7fce440b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.311+0000 7fce4a200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce440d02c0 0x7fce440d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.311+0000 7fce4a200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce440b7da0 con 0x7fce4415cdc0 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.311+0000 7fce4a200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fce440cde60 con 0x7fce440d02c0 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.311+0000 7fce4a200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce440ce080 con 0x7fce440b7450 2024-04-25T19:03:01.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce48e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fce4415cdc0 0x7fce440cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48008/0 (socket says 172.21.15.123:48008) 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce48e00640 1 -- 172.21.15.123:0/2468234811 learned_addr learned my addr 172.21.15.123:0/2468234811 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce49800640 1 --2- 172.21.15.123:0/2468234811 >> v2:172.21.15.123:3300/0 conn(0x7fce440d02c0 0x7fce440d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce43e00640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce440b7450 0x7fce440b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce43400640 1 -- 172.21.15.123:0/2468234811 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4205123339 0 0) 0x7fce3c001610 con 0x7fce4415cdc0 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce43400640 1 -- 172.21.15.123:0/2468234811 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce28003520 con 0x7fce4415cdc0 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce49800640 1 -- 172.21.15.123:0/2468234811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce440b7450 msgr2=0x7fce440b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce49800640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce440b7450 0x7fce440b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce49800640 1 -- 172.21.15.123:0/2468234811 >> v1:172.21.15.123:6789/0 conn(0x7fce4415cdc0 legacy=0x7fce440cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.312+0000 7fce49800640 1 -- 172.21.15.123:0/2468234811 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce440ce360 con 0x7fce440d02c0 2024-04-25T19:03:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.313+0000 7fce49800640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440d02c0 0x7fce440d06d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce340097a0 tx=0x7fce3402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd14df4e565a87a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.314+0000 7fce43400640 1 -- 172.21.15.123:0/2468234811 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34037070 con 0x7fce440d02c0 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.314+0000 7fce43400640 1 -- 172.21.15.123:0/2468234811 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce34032440 con 0x7fce440d02c0 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.314+0000 7fce43400640 1 -- 172.21.15.123:0/2468234811 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34032810 con 0x7fce440d02c0 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.314+0000 7fce4a200640 1 -- 172.21.15.123:0/2468234811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440d02c0 msgr2=0x7fce440d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.314+0000 7fce4a200640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440d02c0 0x7fce440d06d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce340097a0 tx=0x7fce3402a210 comp rx=0 tx=0).stop 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 -- 172.21.15.123:0/2468234811 shutdown_connections 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440d02c0 0x7fce440d06d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 --2- 172.21.15.123:0/2468234811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce440b7450 0x7fce440b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 -- 172.21.15.123:0/2468234811 >> 172.21.15.123:0/2468234811 conn(0x7fce4409b4a0 msgr2=0x7fce4409b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 -- 172.21.15.123:0/2468234811 shutdown_connections 2024-04-25T19:03:01.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.315+0000 7fce4a200640 1 -- 172.21.15.123:0/2468234811 wait complete. 2024-04-25T19:03:01.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.317+0000 7fce4a200640 1 Processor -- start 2024-04-25T19:03:01.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.317+0000 7fce4a200640 1 -- start start 2024-04-25T19:03:01.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce440d02c0 0x7fce441f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce4415cdc0 0x7fce441f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce440cec00 con 0x7fce440b7450 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fce440ce920 con 0x7fce440d02c0 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce4a200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce4415d1d0 con 0x7fce4415cdc0 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43354/0 (socket says 172.21.15.123:43354) 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 -- 172.21.15.123:0/182579090 learned_addr learned my addr 172.21.15.123:0/182579090 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce49800640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce4415cdc0 0x7fce441f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce43e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce440d02c0 0x7fce441f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 -- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce4415cdc0 msgr2=0x7fce441f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce4415cdc0 0x7fce441f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 -- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce440d02c0 msgr2=0x7fce441f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.319+0000 7fce48e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce440d02c0 0x7fce441f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce48e00640 1 -- 172.21.15.123:0/182579090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4409f500 con 0x7fce440b7450 2024-04-25T19:03:01.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce48e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c760 tx=0x7fce3c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c017070 con 0x7fce440b7450 2024-04-25T19:03:01.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c00a640 con 0x7fce440b7450 2024-04-25T19:03:01.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4409f7e0 con 0x7fce440b7450 2024-04-25T19:03:01.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00aa10 con 0x7fce440b7450 2024-04-25T19:03:01.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.320+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce440c62c0 con 0x7fce440b7450 2024-04-25T19:03:01.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.321+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce44050740 con 0x7fce440b7450 2024-04-25T19:03:01.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.323+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce3c01a070 con 0x7fce440b7450 2024-04-25T19:03:01.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.324+0000 7fce40c00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 0x7fce200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.324+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fce3c094c50 con 0x7fce440b7450 2024-04-25T19:03:01.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.324+0000 7fce43e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 0x7fce200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.325+0000 7fce43e00640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 0x7fce200769c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006fd0 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.328+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce440b7da0 con 0x7fce440b7450 2024-04-25T19:03:01.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.528+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce4415c460 con 0x7fce440b7450 2024-04-25T19:03:01.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.529+0000 7fce40c00640 1 -- 172.21.15.123:0/182579090 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7fce3c05f090 con 0x7fce440b7450 2024-04-25T19:03:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:01.532 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 msgr2=0x7fce200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 0x7fce200769c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006fd0 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2024-04-25T19:03:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 msgr2=0x7fce441594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c760 tx=0x7fce3c00cc20 comp rx=0 tx=0).stop 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 shutdown_connections 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.532+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce200744b0 0x7fce200769c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce4415cdc0 0x7fce441f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce440d02c0 0x7fce441f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 --2- 172.21.15.123:0/182579090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce440b7450 0x7fce441594f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 >> 172.21.15.123:0/182579090 conn(0x7fce4409b4a0 msgr2=0x7fce4406f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 shutdown_connections 2024-04-25T19:03:01.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.533+0000 7fce4a200640 1 -- 172.21.15.123:0/182579090 wait complete. 2024-04-25T19:03:01.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T19:03:01.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.837+0000 7f1219c00640 1 Processor -- start 2024-04-25T19:03:01.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.837+0000 7f1219c00640 1 -- start start 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12140cfff0 0x7f121415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 0x7f12140cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f121415e2e0 con 0x7f1214143040 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f12140d1200 con 0x7f12140cfff0 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f12140d05f0 con 0x7f121406dc90 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1213e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12140cfff0 0x7f121415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f120b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1214143040 0x7f121406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48010/0 (socket says 172.21.15.123:48010) 2024-04-25T19:03:01.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f120b400640 1 -- 172.21.15.123:0/2243811161 learned_addr learned my addr 172.21.15.123:0/2243811161 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:01.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.840+0000 7f1219200640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 0x7f12140cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1213e00640 1 -- 172.21.15.123:0/2243811161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 msgr2=0x7f12140cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1213e00640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 0x7f12140cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1213e00640 1 -- 172.21.15.123:0/2243811161 >> v1:172.21.15.123:6789/0 conn(0x7f1214143040 legacy=0x7f121406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1213e00640 1 -- 172.21.15.123:0/2243811161 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140d08d0 con 0x7f12140cfff0 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1213400640 1 -- 172.21.15.123:0/2243811161 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513214634 0 0) 0x7f11fc001490 con 0x7f1214143040 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.841+0000 7f1219200640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 0x7f12140cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.842+0000 7f1213e00640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f121415dda0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced2076742833d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.842+0000 7f1213400640 1 -- 172.21.15.123:0/2243811161 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204037070 con 0x7f12140cfff0 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.842+0000 7f1213400640 1 -- 172.21.15.123:0/2243811161 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204032440 con 0x7f12140cfff0 2024-04-25T19:03:01.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.842+0000 7f1213400640 1 -- 172.21.15.123:0/2243811161 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204032810 con 0x7f12140cfff0 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 -- 172.21.15.123:0/2243811161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 msgr2=0x7f121415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f121415dda0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12040097a0 tx=0x7f120402a180 comp rx=0 tx=0).stop 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 -- 172.21.15.123:0/2243811161 shutdown_connections 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f121406dc90 0x7f12140cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 --2- 172.21.15.123:0/2243811161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f121415dda0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 -- 172.21.15.123:0/2243811161 >> 172.21.15.123:0/2243811161 conn(0x7f1214073090 msgr2=0x7f12140734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:01.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 -- 172.21.15.123:0/2243811161 shutdown_connections 2024-04-25T19:03:01.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.843+0000 7f1219c00640 1 -- 172.21.15.123:0/2243811161 wait complete. 2024-04-25T19:03:01.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.845+0000 7f1219c00640 1 Processor -- start 2024-04-25T19:03:01.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.845+0000 7f1219c00640 1 -- start start 2024-04-25T19:03:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 0x7f121415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 0x7f121413fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1214143450 con 0x7f12140cfff0 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f12140d0e90 con 0x7f121415fc30 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1219c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f12141e9d90 con 0x7f1214143040 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1213e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1213e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43384/0 (socket says 172.21.15.123:43384) 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.847+0000 7f1213e00640 1 -- 172.21.15.123:0/2067982341 learned_addr learned my addr 172.21.15.123:0/2067982341 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f120b400640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 0x7f121415f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1219200640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 0x7f121413fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 -- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 msgr2=0x7f121415f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 0x7f121415f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 -- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 msgr2=0x7f121413fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 0x7f121413fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 -- 172.21.15.123:0/2067982341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121413fba0 con 0x7f12140cfff0 2024-04-25T19:03:01.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f120b400640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 0x7f121415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1219200640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 0x7f121413fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1213e00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f12040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204037040 con 0x7f12140cfff0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204004230 con 0x7f12140cfff0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204032e40 con 0x7f12140cfff0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.848+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141417b0 con 0x7f12140cfff0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.849+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1214141d70 con 0x7f12140cfff0 2024-04-25T19:03:01.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.850+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214050740 con 0x7f12140cfff0 2024-04-25T19:03:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.851+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1204030770 con 0x7f12140cfff0 2024-04-25T19:03:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.852+0000 7f1210c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 0x7f11f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:01.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.852+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f12040b6820 con 0x7f12140cfff0 2024-04-25T19:03:01.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.852+0000 7f120b400640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 0x7f11f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:01.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.853+0000 7f120b400640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 0x7f11f00769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:01.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:01.857+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1204080c70 con 0x7f12140cfff0 2024-04-25T19:03:02.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.100+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f12141e77b0 con 0x7f12140cfff0 2024-04-25T19:03:02.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.671+0000 7f1210c00640 1 -- 172.21.15.123:0/2067982341 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f1204080570 con 0x7f12140cfff0 2024-04-25T19:03:02.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 msgr2=0x7f11f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 0x7f11f00769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 msgr2=0x7f12141e9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f12040046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 shutdown_connections 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11f00744b0 0x7f11f00769c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f121415fc30 0x7f121413fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1214143040 0x7f121415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.675+0000 7f1219c00640 1 --2- 172.21.15.123:0/2067982341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12140cfff0 0x7f12141e9400 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.676+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 >> 172.21.15.123:0/2067982341 conn(0x7f1214073090 msgr2=0x7f12141425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.676+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 shutdown_connections 2024-04-25T19:03:02.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.676+0000 7f1219c00640 1 -- 172.21.15.123:0/2067982341 wait complete. 2024-04-25T19:03:02.711 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T19:03:02.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T19:03:02.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.989+0000 7fdf23e00640 1 Processor -- start 2024-04-25T19:03:02.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.989+0000 7fdf23e00640 1 -- start start 2024-04-25T19:03:02.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.991+0000 7fdf23e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf1c15dfc0 0x7fdf1c162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.991+0000 7fdf23e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15e3d0 con 0x7fdf1c0cd140 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.991+0000 7fdf23e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0cc5f0 con 0x7fdf1c06daa0 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.991+0000 7fdf23e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c06deb0 con 0x7fdf1c15dfc0 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.992+0000 7fdf20a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdf1c0cd140 0x7fdf1c0af040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48012/0 (socket says 172.21.15.123:48012) 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.992+0000 7fdf20a00640 1 -- 172.21.15.123:0/2902178141 learned_addr learned my addr 172.21.15.123:0/2902178141 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:02.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.992+0000 7fdf21e00640 1 --2- 172.21.15.123:0/2902178141 >> v2:172.21.15.146:3300/0 conn(0x7fdf1c15dfc0 0x7fdf1c162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.992+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320277161 0 0) 0x7fdf08001110 con 0x7fdf1c0cd140 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.992+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf04003520 con 0x7fdf1c0cd140 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1983206578 0 0) 0x7fdf04003520 con 0x7fdf1c0cd140 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 >> v2:172.21.15.146:3300/0 conn(0x7fdf1c15dfc0 msgr2=0x7fdf1c162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 --2- 172.21.15.123:0/2902178141 >> v2:172.21.15.146:3300/0 conn(0x7fdf1c15dfc0 0x7fdf1c162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 >> v1:172.21.15.146:6789/0 conn(0x7fdf1c06daa0 legacy=0x7fdf1c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c06ca00 con 0x7fdf1c0cd140 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fdf08001840 con 0x7fdf1c0cd140 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 >> v1:172.21.15.123:6789/0 conn(0x7fdf1c0cd140 legacy=0x7fdf1c0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf04003a00 0x7fdf04003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:02.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf04004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf04005270 0x7fdf0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c06deb0 con 0x7fdf04004800 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0400c190 con 0x7fdf04003a00 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0400c390 con 0x7fdf04005270 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf20a00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf04004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf21e00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf04005270 0x7fdf0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.993+0000 7fdf21400640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf04003a00 0x7fdf04003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf20a00640 1 -- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf04005270 msgr2=0x7fdf0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf20a00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf04005270 0x7fdf0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf20a00640 1 -- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf04003a00 msgr2=0x7fdf04003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf20a00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf04003a00 0x7fdf04003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf20a00640 1 -- 172.21.15.123:0/2902178141 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf04003520 con 0x7fdf04004800 2024-04-25T19:03:02.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf21400640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf04003a00 0x7fdf04003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf21e00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf04005270 0x7fdf0400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.994+0000 7fdf21e00640 1 -- 172.21.15.123:0/2902178141 reap_dead start 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.995+0000 7fdf20a00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf04004c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c320 tx=0x7fdf04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a173fe27093c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.995+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08007e10 con 0x7fdf04004800 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.995+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08005ad0 con 0x7fdf04004800 2024-04-25T19:03:02.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.995+0000 7fdf1be00640 1 -- 172.21.15.123:0/2902178141 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08007070 con 0x7fdf04004800 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 -- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 msgr2=0x7fdf04004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf04004c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c320 tx=0x7fdf04005830 comp rx=0 tx=0).stop 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 -- 172.21.15.123:0/2902178141 shutdown_connections 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 --2- 172.21.15.123:0/2902178141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf04004c10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 -- 172.21.15.123:0/2902178141 >> 172.21.15.123:0/2902178141 conn(0x7fdf1c0ce390 msgr2=0x7fdf1c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 -- 172.21.15.123:0/2902178141 shutdown_connections 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.996+0000 7fdf23e00640 1 -- 172.21.15.123:0/2902178141 wait complete. 2024-04-25T19:03:03.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.998+0000 7fdf23e00640 1 Processor -- start 2024-04-25T19:03:03.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:02.999+0000 7fdf23e00640 1 -- start start 2024-04-25T19:03:03.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf1c14c850 0x7fdf1c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 0x7fdf1c160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c06cfc0 con 0x7fdf04004800 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c06d610 con 0x7fdf1c14c850 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf23e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c06d2a0 con 0x7fdf1c1602f0 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.000+0000 7fdf21400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43408/0 (socket says 172.21.15.123:43408) 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 -- 172.21.15.123:0/3836430000 learned_addr learned my addr 172.21.15.123:0/3836430000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf20a00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf1c14c850 0x7fdf1c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 0x7fdf1c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 -- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 msgr2=0x7fdf1c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 0x7fdf1c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 -- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf1c14c850 msgr2=0x7fdf1c146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf1c14c850 0x7fdf1c146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 -- 172.21.15.123:0/3836430000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1466d0 con 0x7fdf04004800 2024-04-25T19:03:03.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 0x7fdf1c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.001+0000 7fdf21400640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000d770 tx=0x7fdf1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.002+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10005270 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.002+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf10017070 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.002+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c160c40 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.002+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000b6b0 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.002+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c153ed0 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.003+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c076a40 con 0x7fdf04004800 2024-04-25T19:03:03.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.004+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf1000b850 con 0x7fdf04004800 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.005+0000 7fdf19600640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.005+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdf100953e0 con 0x7fdf04004800 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.005+0000 7fdf20a00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.006+0000 7fdf20a00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 0x7fdefc0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf08033000 tx=0x7fdf0802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.010+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf1c160c40 con 0x7fdf04004800 2024-04-25T19:03:03.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.250+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdf1c069c60 con 0x7fdf04004800 2024-04-25T19:03:03.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.675+0000 7fdf19600640 1 -- 172.21.15.123:0/3836430000 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fdf1c1466d0 con 0x7fdf04004800 2024-04-25T19:03:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 msgr2=0x7fdefc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 0x7fdefc0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf08033000 tx=0x7fdf0802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 msgr2=0x7fdf1c154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000d770 tx=0x7fdf1000dc30 comp rx=0 tx=0).stop 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 shutdown_connections 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf1c1602f0 0x7fdf1c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf1c14c850 0x7fdf1c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 --2- 172.21.15.123:0/3836430000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04004800 0x7fdf1c154620 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 >> 172.21.15.123:0/3836430000 conn(0x7fdf1c0ce390 msgr2=0x7fdf1c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.678+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 shutdown_connections 2024-04-25T19:03:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.679+0000 7fdf23e00640 1 -- 172.21.15.123:0/3836430000 wait complete. 2024-04-25T19:03:03.712 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T19:03:03.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:03:03.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.974+0000 7ff39d800640 1 Processor -- start 2024-04-25T19:03:03.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.974+0000 7ff39d800640 1 -- start start 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff39d800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3980d0e60 0x7ff398160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff39d800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39806c710 con 0x7ff3980b4b30 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff39d800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39806c9f0 con 0x7ff3980d4d40 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff39d800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff3980d5150 con 0x7ff3980d0e60 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff397e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff3980b4b30 0x7ff3980b8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48022/0 (socket says 172.21.15.123:48022) 2024-04-25T19:03:03.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.976+0000 7ff397e00640 1 -- 172.21.15.123:0/2678504169 learned_addr learned my addr 172.21.15.123:0/2678504169 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff39ce00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3980d0e60 0x7ff398160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309982895 0 0) 0x7ff39806c710 con 0x7ff3980b4b30 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c003520 con 0x7ff3980b4b30 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1908694051 0 0) 0x7ff37c003520 con 0x7ff3980b4b30 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3980d0e60 msgr2=0x7ff398160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3980d0e60 0x7ff398160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 >> v1:172.21.15.146:6789/0 conn(0x7ff3980d4d40 legacy=0x7ff3980d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39806ccd0 con 0x7ff3980b4b30 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7ff388001670 con 0x7ff3980b4b30 2024-04-25T19:03:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 >> v1:172.21.15.123:6789/0 conn(0x7ff3980b4b30 legacy=0x7ff3980b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff37c004800 0x7ff37c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0051a0 0x7ff37c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980d5150 con 0x7ff37c0039e0 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c00bf30 con 0x7ff37c004800 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c00c130 con 0x7ff37c0051a0 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.977+0000 7ff397e00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397400640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff37c004800 0x7ff37c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff39ce00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0051a0 0x7ff37c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397e00640 1 -- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0051a0 msgr2=0x7ff37c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397e00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0051a0 0x7ff37c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397e00640 1 -- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff37c004800 msgr2=0x7ff37c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397e00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff37c004800 0x7ff37c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.978+0000 7ff397e00640 1 -- 172.21.15.123:0/2678504169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c003520 con 0x7ff37c0039e0 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.979+0000 7ff397e00640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0057e0 tx=0x7ff37c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d8a00db685d478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.979+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007d10 con 0x7ff37c0039e0 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.979+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388005ad0 con 0x7ff37c0039e0 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.979+0000 7ff396a00640 1 -- 172.21.15.123:0/2678504169 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007070 con 0x7ff37c0039e0 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 -- 172.21.15.123:0/2678504169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c0039e0 msgr2=0x7ff37c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 --2- 172.21.15.123:0/2678504169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0057e0 tx=0x7ff37c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39ce00640 1 -- 172.21.15.123:0/2678504169 reap_dead start 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 -- 172.21.15.123:0/2678504169 shutdown_connections 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 -- 172.21.15.123:0/2678504169 >> 172.21.15.123:0/2678504169 conn(0x7ff3980d65d0 msgr2=0x7ff3980b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 -- 172.21.15.123:0/2678504169 shutdown_connections 2024-04-25T19:03:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.980+0000 7ff39d800640 1 -- 172.21.15.123:0/2678504169 wait complete. 2024-04-25T19:03:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.983+0000 7ff39d800640 1 Processor -- start 2024-04-25T19:03:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.983+0000 7ff39d800640 1 -- start start 2024-04-25T19:03:03.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0039e0 0x7ff3980c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3980cd340 0x7ff3980b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980b95e0 con 0x7ff3980ce740 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980d1410 con 0x7ff3980cd340 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.984+0000 7ff39d800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff3980d0e60 con 0x7ff37c0039e0 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43414/0 (socket says 172.21.15.123:43414) 2024-04-25T19:03:03.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 -- 172.21.15.123:0/3568020104 learned_addr learned my addr 172.21.15.123:0/3568020104 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397e00640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0039e0 0x7ff3980c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff39ce00640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3980cd340 0x7ff3980b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 -- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0039e0 msgr2=0x7ff3980c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0039e0 0x7ff3980c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 -- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3980cd340 msgr2=0x7ff3980b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3980cd340 0x7ff3980b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 -- 172.21.15.123:0/3568020104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39815a6a0 con 0x7ff3980ce740 2024-04-25T19:03:03.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.985+0000 7ff397400640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e4d0 tx=0x7ff38000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.986+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800052e0 con 0x7ff3980ce740 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.986+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff380018070 con 0x7ff3980ce740 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.986+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3980c86e0 con 0x7ff3980ce740 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.986+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380003810 con 0x7ff3980ce740 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.986+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3980c9a90 con 0x7ff3980ce740 2024-04-25T19:03:03.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.987+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3980c9520 con 0x7ff3980ce740 2024-04-25T19:03:03.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.988+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3800039b0 con 0x7ff3980ce740 2024-04-25T19:03:03.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.989+0000 7ff38fe00640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 0x7ff3740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.989+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff380095240 con 0x7ff3980ce740 2024-04-25T19:03:03.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.989+0000 7ff397e00640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 0x7ff3740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.990+0000 7ff397e00640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 0x7ff3740769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff388032000 tx=0x7ff388009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:03.994+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff39815a6a0 con 0x7ff3980ce740 2024-04-25T19:03:04.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.235+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3980b87f0 con 0x7ff3980ce740 2024-04-25T19:03:04.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.237+0000 7ff38fe00640 1 -- 172.21.15.123:0/3568020104 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7ff38005f680 con 0x7ff3980ce740 2024-04-25T19:03:04.238 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:03:04.239 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:04.239 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 msgr2=0x7ff3740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 0x7ff3740769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff388032000 tx=0x7ff388009030 comp rx=0 tx=0).stop 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 msgr2=0x7ff3980cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e4d0 tx=0x7ff38000e990 comp rx=0 tx=0).stop 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 shutdown_connections 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff3740744e0 0x7ff3740769f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3980cd340 0x7ff3980b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3980ce740 0x7ff3980cddb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 --2- 172.21.15.123:0/3568020104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0039e0 0x7ff3980c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 >> 172.21.15.123:0/3568020104 conn(0x7ff3980d65d0 msgr2=0x7ff3981eb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 shutdown_connections 2024-04-25T19:03:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.239+0000 7ff39d800640 1 -- 172.21.15.123:0/3568020104 wait complete. 2024-04-25T19:03:04.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:04.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.526+0000 7f5f3a000640 1 Processor -- start 2024-04-25T19:03:04.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.527+0000 7f5f3a000640 1 -- start start 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f3406fe20 0x7f5f3415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 0x7f5f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f3415e2f0 0x7f5f3415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3406ded0 con 0x7f5f3415e2f0 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3415f070 con 0x7f5f3406fe20 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f3a000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3406f340 con 0x7f5f3406d580 2024-04-25T19:03:04.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f39600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f3415e2f0 0x7f5f3415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f39600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f3415e2f0 0x7f5f3415e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43430/0 (socket says 172.21.15.123:43430) 2024-04-25T19:03:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.529+0000 7f5f39600640 1 -- 172.21.15.123:0/3692271362 learned_addr learned my addr 172.21.15.123:0/3692271362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f38c00640 1 --2- 172.21.15.123:0/3692271362 >> v2:172.21.15.146:3300/0 conn(0x7f5f3406fe20 0x7f5f3415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f33e00640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 0x7f5f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f39600640 1 -- 172.21.15.123:0/3692271362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 msgr2=0x7f5f3406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f39600640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 0x7f5f3406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f39600640 1 -- 172.21.15.123:0/3692271362 >> v2:172.21.15.146:3300/0 conn(0x7f5f3406fe20 msgr2=0x7f5f3415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f39600640 1 --2- 172.21.15.123:0/3692271362 >> v2:172.21.15.146:3300/0 conn(0x7f5f3406fe20 0x7f5f3415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f39600640 1 -- 172.21.15.123:0/3692271362 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3406f620 con 0x7f5f3415e2f0 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.530+0000 7f5f33e00640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 0x7f5f3406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.531+0000 7f5f39600640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f3415e700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f34074310 tx=0x7f5f2c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb795c6c4b630a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.531+0000 7f5f33400640 1 -- 172.21.15.123:0/3692271362 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037070 con 0x7f5f3415e2f0 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.531+0000 7f5f33400640 1 -- 172.21.15.123:0/3692271362 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c032440 con 0x7f5f3415e2f0 2024-04-25T19:03:04.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.531+0000 7f5f33400640 1 -- 172.21.15.123:0/3692271362 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c032810 con 0x7f5f3415e2f0 2024-04-25T19:03:04.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 -- 172.21.15.123:0/3692271362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 msgr2=0x7f5f3415e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f3415e700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f34074310 tx=0x7f5f2c02a140 comp rx=0 tx=0).stop 2024-04-25T19:03:04.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 -- 172.21.15.123:0/3692271362 shutdown_connections 2024-04-25T19:03:04.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f3415e700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 --2- 172.21.15.123:0/3692271362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406d580 0x7f5f3406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 --2- 172.21.15.123:0/3692271362 >> v2:172.21.15.146:3300/0 conn(0x7f5f3406fe20 0x7f5f3415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 -- 172.21.15.123:0/3692271362 >> 172.21.15.123:0/3692271362 conn(0x7f5f34073200 msgr2=0x7f5f34073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 -- 172.21.15.123:0/3692271362 shutdown_connections 2024-04-25T19:03:04.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.532+0000 7f5f3a000640 1 -- 172.21.15.123:0/3692271362 wait complete. 2024-04-25T19:03:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.534+0000 7f5f3a000640 1 Processor -- start 2024-04-25T19:03:04.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.535+0000 7f5f3a000640 1 -- start start 2024-04-25T19:03:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3406d580 0x7f5f3415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406fe20 0x7f5f341f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415ec40 con 0x7f5f3415e2f0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415bbf0 con 0x7f5f3406d580 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f3a000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3415edc0 con 0x7f5f3406fe20 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f39600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f39600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43438/0 (socket says 172.21.15.123:43438) 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.536+0000 7f5f39600640 1 -- 172.21.15.123:0/55445904 learned_addr learned my addr 172.21.15.123:0/55445904 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f33e00640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406fe20 0x7f5f341f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f38c00640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3406d580 0x7f5f3415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 -- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406fe20 msgr2=0x7f5f341f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406fe20 0x7f5f341f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 -- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3406d580 msgr2=0x7f5f3415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3406d580 0x7f5f3415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 -- 172.21.15.123:0/55445904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341fa5f0 con 0x7f5f3415e2f0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f39600640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02a180 tx=0x7f5f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:04.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037040 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c042070 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f34144480 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.537+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c030920 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.538+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f34144a40 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.539+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f341fa360 con 0x7f5f3415e2f0 2024-04-25T19:03:04.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.540+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f2c030ac0 con 0x7f5f3415e2f0 2024-04-25T19:03:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.541+0000 7f5f30c00640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 0x7f5f140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.542+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5f2c0b5f90 con 0x7f5f3415e2f0 2024-04-25T19:03:04.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.542+0000 7f5f38c00640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 0x7f5f140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:04.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.542+0000 7f5f38c00640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 0x7f5f140769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f340742c0 tx=0x7f5f24003cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:04.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.546+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f2c080340 con 0x7f5f3415e2f0 2024-04-25T19:03:04.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.742+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f34050740 con 0x7f5f3415e2f0 2024-04-25T19:03:04.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.745+0000 7f5f30c00640 1 -- 172.21.15.123:0/55445904 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f5f2c07fc40 con 0x7f5f3415e2f0 2024-04-25T19:03:04.748 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:04.748 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 msgr2=0x7f5f140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 0x7f5f140769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f340742c0 tx=0x7f5f24003cd0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 msgr2=0x7f5f340ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02a180 tx=0x7f5f2c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 shutdown_connections 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5f140744b0 0x7f5f140769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f3415e2f0 0x7f5f340ced00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f3406fe20 0x7f5f341f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 --2- 172.21.15.123:0/55445904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3406d580 0x7f5f3415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 >> 172.21.15.123:0/55445904 conn(0x7f5f34073200 msgr2=0x7f5f3415f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 shutdown_connections 2024-04-25T19:03:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:04.748+0000 7f5f3a000640 1 -- 172.21.15.123:0/55445904 wait complete. 2024-04-25T19:03:04.773 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T19:03:04.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:03:05.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.055+0000 7f7eb7600640 1 Processor -- start 2024-04-25T19:03:05.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.055+0000 7f7eb7600640 1 -- start start 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb7600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7eb015d7a0 0x7f7eb006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb7600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7eb006d340 0x7f7eb006f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb7600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb006d750 con 0x7f7eb0156cd0 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb7600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb015dbb0 con 0x7f7eb015d7a0 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb7600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb00afe00 con 0x7f7eb006d340 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7ea7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7eb015d7a0 0x7f7eb006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb4c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7eb0156cd0 0x7f7eb0070e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48028/0 (socket says 172.21.15.123:48028) 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.057+0000 7f7eb4c00640 1 -- 172.21.15.123:0/3040807840 learned_addr learned my addr 172.21.15.123:0/3040807840 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7eb5600640 1 --2- 172.21.15.123:0/3040807840 >> v2:172.21.15.146:3300/0 conn(0x7f7eb006d340 0x7f7eb006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7ea7e00640 1 -- 172.21.15.123:0/3040807840 >> v2:172.21.15.146:3300/0 conn(0x7f7eb006d340 msgr2=0x7f7eb006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/3040807840 >> v2:172.21.15.146:3300/0 conn(0x7f7eb006d340 0x7f7eb006f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7ea7e00640 1 -- 172.21.15.123:0/3040807840 >> v1:172.21.15.123:6789/0 conn(0x7f7eb0156cd0 legacy=0x7f7eb0070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7ea7e00640 1 -- 172.21.15.123:0/3040807840 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb00b00a0 con 0x7f7eb015d7a0 2024-04-25T19:03:05.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.058+0000 7f7eafe00640 1 -- 172.21.15.123:0/3040807840 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735183328 0 0) 0x7f7ea0001110 con 0x7f7eb0156cd0 2024-04-25T19:03:05.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.059+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/3040807840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb015d7a0 0x7f7eb006ce00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00715b0 tx=0x7f7e9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0d489bb2003724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.059+0000 7f7eafe00640 1 -- 172.21.15.123:0/3040807840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98037070 con 0x7f7eb015d7a0 2024-04-25T19:03:05.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.059+0000 7f7eafe00640 1 -- 172.21.15.123:0/3040807840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98032440 con 0x7f7eb015d7a0 2024-04-25T19:03:05.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.060+0000 7f7eafe00640 1 -- 172.21.15.123:0/3040807840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9802ae60 con 0x7f7eb015d7a0 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 -- 172.21.15.123:0/3040807840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb015d7a0 msgr2=0x7f7eb006ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 --2- 172.21.15.123:0/3040807840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb015d7a0 0x7f7eb006ce00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00715b0 tx=0x7f7e9802a110 comp rx=0 tx=0).stop 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 -- 172.21.15.123:0/3040807840 shutdown_connections 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 --2- 172.21.15.123:0/3040807840 >> v2:172.21.15.146:3300/0 conn(0x7f7eb006d340 0x7f7eb006f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 --2- 172.21.15.123:0/3040807840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb015d7a0 0x7f7eb006ce00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 -- 172.21.15.123:0/3040807840 >> 172.21.15.123:0/3040807840 conn(0x7f7eb00d1ae0 msgr2=0x7f7eb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 -- 172.21.15.123:0/3040807840 shutdown_connections 2024-04-25T19:03:05.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.061+0000 7f7eb7600640 1 -- 172.21.15.123:0/3040807840 wait complete. 2024-04-25T19:03:05.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.063+0000 7f7eb7600640 1 Processor -- start 2024-04-25T19:03:05.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.064+0000 7f7eb7600640 1 -- start start 2024-04-25T19:03:05.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 0x7f7eb01420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb015d7a0 0x7f7eb014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b0940 con 0x7f7eb006d340 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b0660 con 0x7f7eb0156cd0 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb7600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb00b0e60 con 0x7f7eb015d7a0 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43466/0 (socket says 172.21.15.123:43466) 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.065+0000 7f7eb4c00640 1 -- 172.21.15.123:0/783239918 learned_addr learned my addr 172.21.15.123:0/783239918 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 0x7f7eb01420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb5600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb015d7a0 0x7f7eb014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 -- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb015d7a0 msgr2=0x7f7eb014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb015d7a0 0x7f7eb014a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 -- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 msgr2=0x7f7eb01420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 0x7f7eb01420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 -- 172.21.15.123:0/783239918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0152fc0 con 0x7f7eb006d340 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 0x7f7eb01420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.066+0000 7f7eb4c00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c440 tx=0x7f7ea000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.067+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0018070 con 0x7f7eb006d340 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.067+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00074e0 con 0x7f7eb006d340 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.067+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0004a20 con 0x7f7eb006d340 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.067+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01532a0 con 0x7f7eb006d340 2024-04-25T19:03:05.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.067+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb0153860 con 0x7f7eb006d340 2024-04-25T19:03:05.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.068+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb0076a40 con 0x7f7eb006d340 2024-04-25T19:03:05.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.069+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ea001b070 con 0x7f7eb006d340 2024-04-25T19:03:05.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.071+0000 7f7ead600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.071+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7ea00948c0 con 0x7f7eb006d340 2024-04-25T19:03:05.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.071+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.072+0000 7f7ea7e00640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e98034000 tx=0x7f7e9802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.076+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eb006d750 con 0x7f7eb006d340 2024-04-25T19:03:05.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.276+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7eb00b61d0 con 0x7f7e8c0744e0 2024-04-25T19:03:05.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.289+0000 7f7ead600640 1 -- 172.21.15.123:0/783239918 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332559 (secure 0 0 0) 0x7f7eb00b61d0 con 0x7f7e8c0744e0 2024-04-25T19:03:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:05.298 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 msgr2=0x7f7e8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e98034000 tx=0x7f7e9802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 msgr2=0x7f7eb00b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c440 tx=0x7f7ea000c900 comp rx=0 tx=0).stop 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 shutdown_connections 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eb015d7a0 0x7f7eb014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.297+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eb0156cd0 0x7f7eb01420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.298+0000 7f7eb7600640 1 --2- 172.21.15.123:0/783239918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7eb006d340 0x7f7eb00b2a50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.298+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 >> 172.21.15.123:0/783239918 conn(0x7f7eb00d1ae0 msgr2=0x7f7eb00b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.298+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 shutdown_connections 2024-04-25T19:03:05.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.299+0000 7f7eb7600640 1 -- 172.21.15.123:0/783239918 wait complete. 2024-04-25T19:03:05.329 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T19:03:04.971011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219516,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755204,"statfs":{"total":773094113280,"available":772869328896,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006129"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137574+0000","last_change":"2024-04-25T19:02:39.052814+0000","last_active":"2024-04-25T19:02:45.137574+0000","last_peered":"2024-04-25T19:02:45.137574+0000","last_clean":"2024-04-25T19:02:45.137574+0000","last_became_active":"2024-04-25T19:02:39.052399+0000","last_became_peered":"2024-04-25T19:02:39.052399+0000","last_unstale":"2024-04-25T19:02:45.137574+0000","last_undegraded":"2024-04-25T19:02:45.137574+0000","last_fullsized":"2024-04-25T19:02:45.137574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202717+0000","last_change":"2024-04-25T19:02:40.072630+0000","last_active":"2024-04-25T19:02:45.202717+0000","last_peered":"2024-04-25T19:02:45.202717+0000","last_clean":"2024-04-25T19:02:45.202717+0000","last_became_active":"2024-04-25T19:02:40.072046+0000","last_became_peered":"2024-04-25T19:02:40.072046+0000","last_unstale":"2024-04-25T19:02:45.202717+0000","last_undegraded":"2024-04-25T19:02:45.202717+0000","last_fullsized":"2024-04-25T19:02:45.202717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200723+0000","last_change":"2024-04-25T19:02:44.161105+0000","last_active":"2024-04-25T19:02:45.200723+0000","last_peered":"2024-04-25T19:02:45.200723+0000","last_clean":"2024-04-25T19:02:45.200723+0000","last_became_active":"2024-04-25T19:02:44.160613+0000","last_became_peered":"2024-04-25T19:02:44.160613+0000","last_unstale":"2024-04-25T19:02:45.200723+0000","last_undegraded":"2024-04-25T19:02:45.200723+0000","last_fullsized":"2024-04-25T19:02:45.200723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:38.319660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203032+0000","last_change":"2024-04-25T19:02:39.042391+0000","last_active":"2024-04-25T19:02:45.203032+0000","last_peered":"2024-04-25T19:02:45.203032+0000","last_clean":"2024-04-25T19:02:45.203032+0000","last_became_active":"2024-04-25T19:02:39.042017+0000","last_became_peered":"2024-04-25T19:02:39.042017+0000","last_unstale":"2024-04-25T19:02:45.203032+0000","last_undegraded":"2024-04-25T19:02:45.203032+0000","last_fullsized":"2024-04-25T19:02:45.203032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203099+0000","last_change":"2024-04-25T19:02:40.073983+0000","last_active":"2024-04-25T19:02:45.203099+0000","last_peered":"2024-04-25T19:02:45.203099+0000","last_clean":"2024-04-25T19:02:45.203099+0000","last_became_active":"2024-04-25T19:02:40.069080+0000","last_became_peered":"2024-04-25T19:02:40.069080+0000","last_unstale":"2024-04-25T19:02:45.203099+0000","last_undegraded":"2024-04-25T19:02:45.203099+0000","last_fullsized":"2024-04-25T19:02:45.203099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198316+0000","last_change":"2024-04-25T19:02:44.148887+0000","last_active":"2024-04-25T19:02:45.198316+0000","last_peered":"2024-04-25T19:02:45.198316+0000","last_clean":"2024-04-25T19:02:45.198316+0000","last_became_active":"2024-04-25T19:02:44.148199+0000","last_became_peered":"2024-04-25T19:02:44.148199+0000","last_unstale":"2024-04-25T19:02:45.198316+0000","last_undegraded":"2024-04-25T19:02:45.198316+0000","last_fullsized":"2024-04-25T19:02:45.198316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:55.854080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199269+0000","last_change":"2024-04-25T19:02:39.047541+0000","last_active":"2024-04-25T19:02:45.199269+0000","last_peered":"2024-04-25T19:02:45.199269+0000","last_clean":"2024-04-25T19:02:45.199269+0000","last_became_active":"2024-04-25T19:02:39.047208+0000","last_became_peered":"2024-04-25T19:02:39.047208+0000","last_unstale":"2024-04-25T19:02:45.199269+0000","last_undegraded":"2024-04-25T19:02:45.199269+0000","last_fullsized":"2024-04-25T19:02:45.199269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200023+0000","last_change":"2024-04-25T19:02:40.076534+0000","last_active":"2024-04-25T19:02:45.200023+0000","last_peered":"2024-04-25T19:02:45.200023+0000","last_clean":"2024-04-25T19:02:45.200023+0000","last_became_active":"2024-04-25T19:02:40.076129+0000","last_became_peered":"2024-04-25T19:02:40.076129+0000","last_unstale":"2024-04-25T19:02:45.200023+0000","last_undegraded":"2024-04-25T19:02:45.200023+0000","last_fullsized":"2024-04-25T19:02:45.200023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198683+0000","last_change":"2024-04-25T19:02:44.153635+0000","last_active":"2024-04-25T19:02:45.198683+0000","last_peered":"2024-04-25T19:02:45.198683+0000","last_clean":"2024-04-25T19:02:45.198683+0000","last_became_active":"2024-04-25T19:02:44.153134+0000","last_became_peered":"2024-04-25T19:02:44.153134+0000","last_unstale":"2024-04-25T19:02:45.198683+0000","last_undegraded":"2024-04-25T19:02:45.198683+0000","last_fullsized":"2024-04-25T19:02:45.198683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:25.102590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199200+0000","last_change":"2024-04-25T19:02:39.047906+0000","last_active":"2024-04-25T19:02:45.199200+0000","last_peered":"2024-04-25T19:02:45.199200+0000","last_clean":"2024-04-25T19:02:45.199200+0000","last_became_active":"2024-04-25T19:02:39.044722+0000","last_became_peered":"2024-04-25T19:02:39.044722+0000","last_unstale":"2024-04-25T19:02:45.199200+0000","last_undegraded":"2024-04-25T19:02:45.199200+0000","last_fullsized":"2024-04-25T19:02:45.199200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130532+0000","last_change":"2024-04-25T19:02:40.084937+0000","last_active":"2024-04-25T19:02:45.130532+0000","last_peered":"2024-04-25T19:02:45.130532+0000","last_clean":"2024-04-25T19:02:45.130532+0000","last_became_active":"2024-04-25T19:02:40.084547+0000","last_became_peered":"2024-04-25T19:02:40.084547+0000","last_unstale":"2024-04-25T19:02:45.130532+0000","last_undegraded":"2024-04-25T19:02:45.130532+0000","last_fullsized":"2024-04-25T19:02:45.130532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:00.967270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197619+0000","last_change":"2024-04-25T19:02:44.152662+0000","last_active":"2024-04-25T19:02:45.197619+0000","last_peered":"2024-04-25T19:02:45.197619+0000","last_clean":"2024-04-25T19:02:45.197619+0000","last_became_active":"2024-04-25T19:02:44.152150+0000","last_became_peered":"2024-04-25T19:02:44.152150+0000","last_unstale":"2024-04-25T19:02:45.197619+0000","last_undegraded":"2024-04-25T19:02:45.197619+0000","last_fullsized":"2024-04-25T19:02:45.197619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:04.766506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129234+0000","last_change":"2024-04-25T19:02:40.069483+0000","last_active":"2024-04-25T19:02:45.129234+0000","last_peered":"2024-04-25T19:02:45.129234+0000","last_clean":"2024-04-25T19:02:45.129234+0000","last_became_active":"2024-04-25T19:02:40.068633+0000","last_became_peered":"2024-04-25T19:02:40.068633+0000","last_unstale":"2024-04-25T19:02:45.129234+0000","last_undegraded":"2024-04-25T19:02:45.129234+0000","last_fullsized":"2024-04-25T19:02:45.129234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:46.741238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130329+0000","last_change":"2024-04-25T19:02:39.036777+0000","last_active":"2024-04-25T19:02:45.130329+0000","last_peered":"2024-04-25T19:02:45.130329+0000","last_clean":"2024-04-25T19:02:45.130329+0000","last_became_active":"2024-04-25T19:02:39.036471+0000","last_became_peered":"2024-04-25T19:02:39.036471+0000","last_unstale":"2024-04-25T19:02:45.130329+0000","last_undegraded":"2024-04-25T19:02:45.130329+0000","last_fullsized":"2024-04-25T19:02:45.130329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:41.460879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130171+0000","last_change":"2024-04-25T19:02:44.153484+0000","last_active":"2024-04-25T19:02:45.130171+0000","last_peered":"2024-04-25T19:02:45.130171+0000","last_clean":"2024-04-25T19:02:45.130171+0000","last_became_active":"2024-04-25T19:02:44.153061+0000","last_became_peered":"2024-04-25T19:02:44.153061+0000","last_unstale":"2024-04-25T19:02:45.130171+0000","last_undegraded":"2024-04-25T19:02:45.130171+0000","last_fullsized":"2024-04-25T19:02:45.130171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:14.106695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201441+0000","last_change":"2024-04-25T19:02:40.071684+0000","last_active":"2024-04-25T19:02:45.201441+0000","last_peered":"2024-04-25T19:02:45.201441+0000","last_clean":"2024-04-25T19:02:45.201441+0000","last_became_active":"2024-04-25T19:02:40.071149+0000","last_became_peered":"2024-04-25T19:02:40.071149+0000","last_unstale":"2024-04-25T19:02:45.201441+0000","last_undegraded":"2024-04-25T19:02:45.201441+0000","last_fullsized":"2024-04-25T19:02:45.201441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134616+0000","last_change":"2024-04-25T19:02:39.047265+0000","last_active":"2024-04-25T19:02:45.134616+0000","last_peered":"2024-04-25T19:02:45.134616+0000","last_clean":"2024-04-25T19:02:45.134616+0000","last_became_active":"2024-04-25T19:02:39.046336+0000","last_became_peered":"2024-04-25T19:02:39.046336+0000","last_unstale":"2024-04-25T19:02:45.134616+0000","last_undegraded":"2024-04-25T19:02:45.134616+0000","last_fullsized":"2024-04-25T19:02:45.134616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:02.247449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198218+0000","last_change":"2024-04-25T19:02:44.139798+0000","last_active":"2024-04-25T19:02:45.198218+0000","last_peered":"2024-04-25T19:02:45.198218+0000","last_clean":"2024-04-25T19:02:45.198218+0000","last_became_active":"2024-04-25T19:02:44.139333+0000","last_became_peered":"2024-04-25T19:02:44.139333+0000","last_unstale":"2024-04-25T19:02:45.198218+0000","last_undegraded":"2024-04-25T19:02:45.198218+0000","last_fullsized":"2024-04-25T19:02:45.198218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:05.053484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131214+0000","last_change":"2024-04-25T19:02:40.065574+0000","last_active":"2024-04-25T19:02:45.131214+0000","last_peered":"2024-04-25T19:02:45.131214+0000","last_clean":"2024-04-25T19:02:45.131214+0000","last_became_active":"2024-04-25T19:02:40.060953+0000","last_became_peered":"2024-04-25T19:02:40.060953+0000","last_unstale":"2024-04-25T19:02:45.131214+0000","last_undegraded":"2024-04-25T19:02:45.131214+0000","last_fullsized":"2024-04-25T19:02:45.131214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132354+0000","last_change":"2024-04-25T19:02:39.044906+0000","last_active":"2024-04-25T19:02:45.132354+0000","last_peered":"2024-04-25T19:02:45.132354+0000","last_clean":"2024-04-25T19:02:45.132354+0000","last_became_active":"2024-04-25T19:02:39.043065+0000","last_became_peered":"2024-04-25T19:02:39.043065+0000","last_unstale":"2024-04-25T19:02:45.132354+0000","last_undegraded":"2024-04-25T19:02:45.132354+0000","last_fullsized":"2024-04-25T19:02:45.132354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203780+0000","last_change":"2024-04-25T19:02:44.149122+0000","last_active":"2024-04-25T19:02:45.203780+0000","last_peered":"2024-04-25T19:02:45.203780+0000","last_clean":"2024-04-25T19:02:45.203780+0000","last_became_active":"2024-04-25T19:02:44.148633+0000","last_became_peered":"2024-04-25T19:02:44.148633+0000","last_unstale":"2024-04-25T19:02:45.203780+0000","last_undegraded":"2024-04-25T19:02:45.203780+0000","last_fullsized":"2024-04-25T19:02:45.203780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128923+0000","last_change":"2024-04-25T19:02:40.067343+0000","last_active":"2024-04-25T19:02:45.128923+0000","last_peered":"2024-04-25T19:02:45.128923+0000","last_clean":"2024-04-25T19:02:45.128923+0000","last_became_active":"2024-04-25T19:02:40.066748+0000","last_became_peered":"2024-04-25T19:02:40.066748+0000","last_unstale":"2024-04-25T19:02:45.128923+0000","last_undegraded":"2024-04-25T19:02:45.128923+0000","last_fullsized":"2024-04-25T19:02:45.128923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:18.495508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196634+0000","last_change":"2024-04-25T19:02:39.042622+0000","last_active":"2024-04-25T19:02:45.196634+0000","last_peered":"2024-04-25T19:02:45.196634+0000","last_clean":"2024-04-25T19:02:45.196634+0000","last_became_active":"2024-04-25T19:02:39.042302+0000","last_became_peered":"2024-04-25T19:02:39.042302+0000","last_unstale":"2024-04-25T19:02:45.196634+0000","last_undegraded":"2024-04-25T19:02:45.196634+0000","last_fullsized":"2024-04-25T19:02:45.196634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199162+0000","last_change":"2024-04-25T19:02:44.148027+0000","last_active":"2024-04-25T19:02:45.199162+0000","last_peered":"2024-04-25T19:02:45.199162+0000","last_clean":"2024-04-25T19:02:45.199162+0000","last_became_active":"2024-04-25T19:02:44.147612+0000","last_became_peered":"2024-04-25T19:02:44.147612+0000","last_unstale":"2024-04-25T19:02:45.199162+0000","last_undegraded":"2024-04-25T19:02:45.199162+0000","last_fullsized":"2024-04-25T19:02:45.199162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:52.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136112+0000","last_change":"2024-04-25T19:02:40.062872+0000","last_active":"2024-04-25T19:02:45.136112+0000","last_peered":"2024-04-25T19:02:45.136112+0000","last_clean":"2024-04-25T19:02:45.136112+0000","last_became_active":"2024-04-25T19:02:40.061519+0000","last_became_peered":"2024-04-25T19:02:40.061519+0000","last_unstale":"2024-04-25T19:02:45.136112+0000","last_undegraded":"2024-04-25T19:02:45.136112+0000","last_fullsized":"2024-04-25T19:02:45.136112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132193+0000","last_change":"2024-04-25T19:02:39.054460+0000","last_active":"2024-04-25T19:02:45.132193+0000","last_peered":"2024-04-25T19:02:45.132193+0000","last_clean":"2024-04-25T19:02:45.132193+0000","last_became_active":"2024-04-25T19:02:39.053233+0000","last_became_peered":"2024-04-25T19:02:39.053233+0000","last_unstale":"2024-04-25T19:02:45.132193+0000","last_undegraded":"2024-04-25T19:02:45.132193+0000","last_fullsized":"2024-04-25T19:02:45.132193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:17.622604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199607+0000","last_change":"2024-04-25T19:02:44.139268+0000","last_active":"2024-04-25T19:02:45.199607+0000","last_peered":"2024-04-25T19:02:45.199607+0000","last_clean":"2024-04-25T19:02:45.199607+0000","last_became_active":"2024-04-25T19:02:44.137641+0000","last_became_peered":"2024-04-25T19:02:44.137641+0000","last_unstale":"2024-04-25T19:02:45.199607+0000","last_undegraded":"2024-04-25T19:02:45.199607+0000","last_fullsized":"2024-04-25T19:02:45.199607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137364+0000","last_change":"2024-04-25T19:02:40.063483+0000","last_active":"2024-04-25T19:02:45.137364+0000","last_peered":"2024-04-25T19:02:45.137364+0000","last_clean":"2024-04-25T19:02:45.137364+0000","last_became_active":"2024-04-25T19:02:40.062531+0000","last_became_peered":"2024-04-25T19:02:40.062531+0000","last_unstale":"2024-04-25T19:02:45.137364+0000","last_undegraded":"2024-04-25T19:02:45.137364+0000","last_fullsized":"2024-04-25T19:02:45.137364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136358+0000","last_change":"2024-04-25T19:02:39.052714+0000","last_active":"2024-04-25T19:02:45.136358+0000","last_peered":"2024-04-25T19:02:45.136358+0000","last_clean":"2024-04-25T19:02:45.136358+0000","last_became_active":"2024-04-25T19:02:39.051543+0000","last_became_peered":"2024-04-25T19:02:39.051543+0000","last_unstale":"2024-04-25T19:02:45.136358+0000","last_undegraded":"2024-04-25T19:02:45.136358+0000","last_fullsized":"2024-04-25T19:02:45.136358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131504+0000","last_change":"2024-04-25T19:02:44.142273+0000","last_active":"2024-04-25T19:02:45.131504+0000","last_peered":"2024-04-25T19:02:45.131504+0000","last_clean":"2024-04-25T19:02:45.131504+0000","last_became_active":"2024-04-25T19:02:44.140876+0000","last_became_peered":"2024-04-25T19:02:44.140876+0000","last_unstale":"2024-04-25T19:02:45.131504+0000","last_undegraded":"2024-04-25T19:02:45.131504+0000","last_fullsized":"2024-04-25T19:02:45.131504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137451+0000","last_change":"2024-04-25T19:02:40.065602+0000","last_active":"2024-04-25T19:02:45.137451+0000","last_peered":"2024-04-25T19:02:45.137451+0000","last_clean":"2024-04-25T19:02:45.137451+0000","last_became_active":"2024-04-25T19:02:40.065178+0000","last_became_peered":"2024-04-25T19:02:40.065178+0000","last_unstale":"2024-04-25T19:02:45.137451+0000","last_undegraded":"2024-04-25T19:02:45.137451+0000","last_fullsized":"2024-04-25T19:02:45.137451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.101532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131078+0000","last_change":"2024-04-25T19:02:39.054975+0000","last_active":"2024-04-25T19:02:45.131078+0000","last_peered":"2024-04-25T19:02:45.131078+0000","last_clean":"2024-04-25T19:02:45.131078+0000","last_became_active":"2024-04-25T19:02:39.052591+0000","last_became_peered":"2024-04-25T19:02:39.052591+0000","last_unstale":"2024-04-25T19:02:45.131078+0000","last_undegraded":"2024-04-25T19:02:45.131078+0000","last_fullsized":"2024-04-25T19:02:45.131078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:10.725639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135617+0000","last_change":"2024-04-25T19:02:44.144565+0000","last_active":"2024-04-25T19:02:45.135617+0000","last_peered":"2024-04-25T19:02:45.135617+0000","last_clean":"2024-04-25T19:02:45.135617+0000","last_became_active":"2024-04-25T19:02:44.144092+0000","last_became_peered":"2024-04-25T19:02:44.144092+0000","last_unstale":"2024-04-25T19:02:45.135617+0000","last_undegraded":"2024-04-25T19:02:45.135617+0000","last_fullsized":"2024-04-25T19:02:45.135617+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:18.177897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198835+0000","last_change":"2024-04-25T19:02:40.062748+0000","last_active":"2024-04-25T19:02:45.198835+0000","last_peered":"2024-04-25T19:02:45.198835+0000","last_clean":"2024-04-25T19:02:45.198835+0000","last_became_active":"2024-04-25T19:02:40.060618+0000","last_became_peered":"2024-04-25T19:02:40.060618+0000","last_unstale":"2024-04-25T19:02:45.198835+0000","last_undegraded":"2024-04-25T19:02:45.198835+0000","last_fullsized":"2024-04-25T19:02:45.198835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200394+0000","last_change":"2024-04-25T19:02:39.043998+0000","last_active":"2024-04-25T19:02:45.200394+0000","last_peered":"2024-04-25T19:02:45.200394+0000","last_clean":"2024-04-25T19:02:45.200394+0000","last_became_active":"2024-04-25T19:02:39.043623+0000","last_became_peered":"2024-04-25T19:02:39.043623+0000","last_unstale":"2024-04-25T19:02:45.200394+0000","last_undegraded":"2024-04-25T19:02:45.200394+0000","last_fullsized":"2024-04-25T19:02:45.200394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133644+0000","last_change":"2024-04-25T19:02:44.158919+0000","last_active":"2024-04-25T19:02:45.133644+0000","last_peered":"2024-04-25T19:02:45.133644+0000","last_clean":"2024-04-25T19:02:45.133644+0000","last_became_active":"2024-04-25T19:02:44.158358+0000","last_became_peered":"2024-04-25T19:02:44.158358+0000","last_unstale":"2024-04-25T19:02:45.133644+0000","last_undegraded":"2024-04-25T19:02:45.133644+0000","last_fullsized":"2024-04-25T19:02:45.133644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:05.831945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198482+0000","last_change":"2024-04-25T19:02:40.072547+0000","last_active":"2024-04-25T19:02:45.198482+0000","last_peered":"2024-04-25T19:02:45.198482+0000","last_clean":"2024-04-25T19:02:45.198482+0000","last_became_active":"2024-04-25T19:02:40.072124+0000","last_became_peered":"2024-04-25T19:02:40.072124+0000","last_unstale":"2024-04-25T19:02:45.198482+0000","last_undegraded":"2024-04-25T19:02:45.198482+0000","last_fullsized":"2024-04-25T19:02:45.198482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130699+0000","last_change":"2024-04-25T19:02:39.051040+0000","last_active":"2024-04-25T19:02:45.130699+0000","last_peered":"2024-04-25T19:02:45.130699+0000","last_clean":"2024-04-25T19:02:45.130699+0000","last_became_active":"2024-04-25T19:02:39.048647+0000","last_became_peered":"2024-04-25T19:02:39.048647+0000","last_unstale":"2024-04-25T19:02:45.130699+0000","last_undegraded":"2024-04-25T19:02:45.130699+0000","last_fullsized":"2024-04-25T19:02:45.130699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:53.187342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136962+0000","last_change":"2024-04-25T19:02:44.151392+0000","last_active":"2024-04-25T19:02:45.136962+0000","last_peered":"2024-04-25T19:02:45.136962+0000","last_clean":"2024-04-25T19:02:45.136962+0000","last_became_active":"2024-04-25T19:02:44.150074+0000","last_became_peered":"2024-04-25T19:02:44.150074+0000","last_unstale":"2024-04-25T19:02:45.136962+0000","last_undegraded":"2024-04-25T19:02:45.136962+0000","last_fullsized":"2024-04-25T19:02:45.136962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:03.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199109+0000","last_change":"2024-04-25T19:02:40.074602+0000","last_active":"2024-04-25T19:02:45.199109+0000","last_peered":"2024-04-25T19:02:45.199109+0000","last_clean":"2024-04-25T19:02:45.199109+0000","last_became_active":"2024-04-25T19:02:40.070530+0000","last_became_peered":"2024-04-25T19:02:40.070530+0000","last_unstale":"2024-04-25T19:02:45.199109+0000","last_undegraded":"2024-04-25T19:02:45.199109+0000","last_fullsized":"2024-04-25T19:02:45.199109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200966+0000","last_change":"2024-04-25T19:02:39.048859+0000","last_active":"2024-04-25T19:02:45.200966+0000","last_peered":"2024-04-25T19:02:45.200966+0000","last_clean":"2024-04-25T19:02:45.200966+0000","last_became_active":"2024-04-25T19:02:39.048497+0000","last_became_peered":"2024-04-25T19:02:39.048497+0000","last_unstale":"2024-04-25T19:02:45.200966+0000","last_undegraded":"2024-04-25T19:02:45.200966+0000","last_fullsized":"2024-04-25T19:02:45.200966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:13.519715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137567+0000","last_change":"2024-04-25T19:02:44.150754+0000","last_active":"2024-04-25T19:02:45.137567+0000","last_peered":"2024-04-25T19:02:45.137567+0000","last_clean":"2024-04-25T19:02:45.137567+0000","last_became_active":"2024-04-25T19:02:44.150092+0000","last_became_peered":"2024-04-25T19:02:44.150092+0000","last_unstale":"2024-04-25T19:02:45.137567+0000","last_undegraded":"2024-04-25T19:02:45.137567+0000","last_fullsized":"2024-04-25T19:02:45.137567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:45.602374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132688+0000","last_change":"2024-04-25T19:02:40.068029+0000","last_active":"2024-04-25T19:02:45.132688+0000","last_peered":"2024-04-25T19:02:45.132688+0000","last_clean":"2024-04-25T19:02:45.132688+0000","last_became_active":"2024-04-25T19:02:40.066355+0000","last_became_peered":"2024-04-25T19:02:40.066355+0000","last_unstale":"2024-04-25T19:02:45.132688+0000","last_undegraded":"2024-04-25T19:02:45.132688+0000","last_fullsized":"2024-04-25T19:02:45.132688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:09.936302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129192+0000","last_change":"2024-04-25T19:02:39.047056+0000","last_active":"2024-04-25T19:02:45.129192+0000","last_peered":"2024-04-25T19:02:45.129192+0000","last_clean":"2024-04-25T19:02:45.129192+0000","last_became_active":"2024-04-25T19:02:39.046198+0000","last_became_peered":"2024-04-25T19:02:39.046198+0000","last_unstale":"2024-04-25T19:02:45.129192+0000","last_undegraded":"2024-04-25T19:02:45.129192+0000","last_fullsized":"2024-04-25T19:02:45.129192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202550+0000","last_change":"2024-04-25T19:02:44.158508+0000","last_active":"2024-04-25T19:02:45.202550+0000","last_peered":"2024-04-25T19:02:45.202550+0000","last_clean":"2024-04-25T19:02:45.202550+0000","last_became_active":"2024-04-25T19:02:44.157734+0000","last_became_peered":"2024-04-25T19:02:44.157734+0000","last_unstale":"2024-04-25T19:02:45.202550+0000","last_undegraded":"2024-04-25T19:02:45.202550+0000","last_fullsized":"2024-04-25T19:02:45.202550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:42.894635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133801+0000","last_change":"2024-04-25T19:02:40.048821+0000","last_active":"2024-04-25T19:02:45.133801+0000","last_peered":"2024-04-25T19:02:45.133801+0000","last_clean":"2024-04-25T19:02:45.133801+0000","last_became_active":"2024-04-25T19:02:40.048295+0000","last_became_peered":"2024-04-25T19:02:40.048295+0000","last_unstale":"2024-04-25T19:02:45.133801+0000","last_undegraded":"2024-04-25T19:02:45.133801+0000","last_fullsized":"2024-04-25T19:02:45.133801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129911+0000","last_change":"2024-04-25T19:02:39.036175+0000","last_active":"2024-04-25T19:02:45.129911+0000","last_peered":"2024-04-25T19:02:45.129911+0000","last_clean":"2024-04-25T19:02:45.129911+0000","last_became_active":"2024-04-25T19:02:39.035822+0000","last_became_peered":"2024-04-25T19:02:39.035822+0000","last_unstale":"2024-04-25T19:02:45.129911+0000","last_undegraded":"2024-04-25T19:02:45.129911+0000","last_fullsized":"2024-04-25T19:02:45.129911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.306938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197039+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:02:45.197039+0000","last_peered":"2024-04-25T19:02:45.197039+0000","last_clean":"2024-04-25T19:02:45.197039+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:02:45.197039+0000","last_undegraded":"2024-04-25T19:02:45.197039+0000","last_fullsized":"2024-04-25T19:02:45.197039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131234+0000","last_change":"2024-04-25T19:02:44.144534+0000","last_active":"2024-04-25T19:02:45.131234+0000","last_peered":"2024-04-25T19:02:45.131234+0000","last_clean":"2024-04-25T19:02:45.131234+0000","last_became_active":"2024-04-25T19:02:44.143365+0000","last_became_peered":"2024-04-25T19:02:44.143365+0000","last_unstale":"2024-04-25T19:02:45.131234+0000","last_undegraded":"2024-04-25T19:02:45.131234+0000","last_fullsized":"2024-04-25T19:02:45.131234+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:39.667305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130665+0000","last_change":"2024-04-25T19:02:40.082915+0000","last_active":"2024-04-25T19:02:45.130665+0000","last_peered":"2024-04-25T19:02:45.130665+0000","last_clean":"2024-04-25T19:02:45.130665+0000","last_became_active":"2024-04-25T19:02:40.082594+0000","last_became_peered":"2024-04-25T19:02:40.082594+0000","last_unstale":"2024-04-25T19:02:45.130665+0000","last_undegraded":"2024-04-25T19:02:45.130665+0000","last_fullsized":"2024-04-25T19:02:45.130665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:10.588191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137737+0000","last_change":"2024-04-25T19:02:39.052990+0000","last_active":"2024-04-25T19:02:45.137737+0000","last_peered":"2024-04-25T19:02:45.137737+0000","last_clean":"2024-04-25T19:02:45.137737+0000","last_became_active":"2024-04-25T19:02:39.052635+0000","last_became_peered":"2024-04-25T19:02:39.052635+0000","last_unstale":"2024-04-25T19:02:45.137737+0000","last_undegraded":"2024-04-25T19:02:45.137737+0000","last_fullsized":"2024-04-25T19:02:45.137737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.298378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198050+0000","last_change":"2024-04-25T19:02:44.143773+0000","last_active":"2024-04-25T19:02:45.198050+0000","last_peered":"2024-04-25T19:02:45.198050+0000","last_clean":"2024-04-25T19:02:45.198050+0000","last_became_active":"2024-04-25T19:02:44.141602+0000","last_became_peered":"2024-04-25T19:02:44.141602+0000","last_unstale":"2024-04-25T19:02:45.198050+0000","last_undegraded":"2024-04-25T19:02:45.198050+0000","last_fullsized":"2024-04-25T19:02:45.198050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:03.052112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199104+0000","last_change":"2024-04-25T19:02:40.075177+0000","last_active":"2024-04-25T19:02:45.199104+0000","last_peered":"2024-04-25T19:02:45.199104+0000","last_clean":"2024-04-25T19:02:45.199104+0000","last_became_active":"2024-04-25T19:02:40.074512+0000","last_became_peered":"2024-04-25T19:02:40.074512+0000","last_unstale":"2024-04-25T19:02:45.199104+0000","last_undegraded":"2024-04-25T19:02:45.199104+0000","last_fullsized":"2024-04-25T19:02:45.199104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201661+0000","last_change":"2024-04-25T19:02:39.052054+0000","last_active":"2024-04-25T19:02:45.201661+0000","last_peered":"2024-04-25T19:02:45.201661+0000","last_clean":"2024-04-25T19:02:45.201661+0000","last_became_active":"2024-04-25T19:02:39.051646+0000","last_became_peered":"2024-04-25T19:02:39.051646+0000","last_unstale":"2024-04-25T19:02:45.201661+0000","last_undegraded":"2024-04-25T19:02:45.201661+0000","last_fullsized":"2024-04-25T19:02:45.201661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200618+0000","last_change":"2024-04-25T19:02:44.157149+0000","last_active":"2024-04-25T19:02:45.200618+0000","last_peered":"2024-04-25T19:02:45.200618+0000","last_clean":"2024-04-25T19:02:45.200618+0000","last_became_active":"2024-04-25T19:02:44.156656+0000","last_became_peered":"2024-04-25T19:02:44.156656+0000","last_unstale":"2024-04-25T19:02:45.200618+0000","last_undegraded":"2024-04-25T19:02:45.200618+0000","last_fullsized":"2024-04-25T19:02:45.200618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:14.674405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198854+0000","last_change":"2024-04-25T19:02:40.080039+0000","last_active":"2024-04-25T19:02:45.198854+0000","last_peered":"2024-04-25T19:02:45.198854+0000","last_clean":"2024-04-25T19:02:45.198854+0000","last_became_active":"2024-04-25T19:02:40.079725+0000","last_became_peered":"2024-04-25T19:02:40.079725+0000","last_unstale":"2024-04-25T19:02:45.198854+0000","last_undegraded":"2024-04-25T19:02:45.198854+0000","last_fullsized":"2024-04-25T19:02:45.198854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136108+0000","last_change":"2024-04-25T19:02:39.051841+0000","last_active":"2024-04-25T19:02:45.136108+0000","last_peered":"2024-04-25T19:02:45.136108+0000","last_clean":"2024-04-25T19:02:45.136108+0000","last_became_active":"2024-04-25T19:02:39.050695+0000","last_became_peered":"2024-04-25T19:02:39.050695+0000","last_unstale":"2024-04-25T19:02:45.136108+0000","last_undegraded":"2024-04-25T19:02:45.136108+0000","last_fullsized":"2024-04-25T19:02:45.136108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128669+0000","last_change":"2024-04-25T19:02:44.147067+0000","last_active":"2024-04-25T19:02:45.128669+0000","last_peered":"2024-04-25T19:02:45.128669+0000","last_clean":"2024-04-25T19:02:45.128669+0000","last_became_active":"2024-04-25T19:02:44.146048+0000","last_became_peered":"2024-04-25T19:02:44.146048+0000","last_unstale":"2024-04-25T19:02:45.128669+0000","last_undegraded":"2024-04-25T19:02:45.128669+0000","last_fullsized":"2024-04-25T19:02:45.128669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:13.605566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199716+0000","last_change":"2024-04-25T19:02:40.066529+0000","last_active":"2024-04-25T19:02:45.199716+0000","last_peered":"2024-04-25T19:02:45.199716+0000","last_clean":"2024-04-25T19:02:45.199716+0000","last_became_active":"2024-04-25T19:02:40.064814+0000","last_became_peered":"2024-04-25T19:02:40.064814+0000","last_unstale":"2024-04-25T19:02:45.199716+0000","last_undegraded":"2024-04-25T19:02:45.199716+0000","last_fullsized":"2024-04-25T19:02:45.199716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199020+0000","last_change":"2024-04-25T19:02:39.051220+0000","last_active":"2024-04-25T19:02:45.199020+0000","last_peered":"2024-04-25T19:02:45.199020+0000","last_clean":"2024-04-25T19:02:45.199020+0000","last_became_active":"2024-04-25T19:02:39.050911+0000","last_became_peered":"2024-04-25T19:02:39.050911+0000","last_unstale":"2024-04-25T19:02:45.199020+0000","last_undegraded":"2024-04-25T19:02:45.199020+0000","last_fullsized":"2024-04-25T19:02:45.199020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196897+0000","last_change":"2024-04-25T19:02:44.156822+0000","last_active":"2024-04-25T19:02:45.196897+0000","last_peered":"2024-04-25T19:02:45.196897+0000","last_clean":"2024-04-25T19:02:45.196897+0000","last_became_active":"2024-04-25T19:02:44.156379+0000","last_became_peered":"2024-04-25T19:02:44.156379+0000","last_unstale":"2024-04-25T19:02:45.196897+0000","last_undegraded":"2024-04-25T19:02:45.196897+0000","last_fullsized":"2024-04-25T19:02:45.196897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128832+0000","last_change":"2024-04-25T19:02:40.075607+0000","last_active":"2024-04-25T19:02:45.128832+0000","last_peered":"2024-04-25T19:02:45.128832+0000","last_clean":"2024-04-25T19:02:45.128832+0000","last_became_active":"2024-04-25T19:02:40.075267+0000","last_became_peered":"2024-04-25T19:02:40.075267+0000","last_unstale":"2024-04-25T19:02:45.128832+0000","last_undegraded":"2024-04-25T19:02:45.128832+0000","last_fullsized":"2024-04-25T19:02:45.128832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131535+0000","last_change":"2024-04-25T19:02:39.044744+0000","last_active":"2024-04-25T19:02:45.131535+0000","last_peered":"2024-04-25T19:02:45.131535+0000","last_clean":"2024-04-25T19:02:45.131535+0000","last_became_active":"2024-04-25T19:02:39.044372+0000","last_became_peered":"2024-04-25T19:02:39.044372+0000","last_unstale":"2024-04-25T19:02:45.131535+0000","last_undegraded":"2024-04-25T19:02:45.131535+0000","last_fullsized":"2024-04-25T19:02:45.131535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203015+0000","last_change":"2024-04-25T19:02:44.158146+0000","last_active":"2024-04-25T19:02:45.203015+0000","last_peered":"2024-04-25T19:02:45.203015+0000","last_clean":"2024-04-25T19:02:45.203015+0000","last_became_active":"2024-04-25T19:02:44.157017+0000","last_became_peered":"2024-04-25T19:02:44.157017+0000","last_unstale":"2024-04-25T19:02:45.203015+0000","last_undegraded":"2024-04-25T19:02:45.203015+0000","last_fullsized":"2024-04-25T19:02:45.203015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:40.012791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196618+0000","last_change":"2024-04-25T19:02:40.073754+0000","last_active":"2024-04-25T19:02:45.196618+0000","last_peered":"2024-04-25T19:02:45.196618+0000","last_clean":"2024-04-25T19:02:45.196618+0000","last_became_active":"2024-04-25T19:02:40.073500+0000","last_became_peered":"2024-04-25T19:02:40.073500+0000","last_unstale":"2024-04-25T19:02:45.196618+0000","last_undegraded":"2024-04-25T19:02:45.196618+0000","last_fullsized":"2024-04-25T19:02:45.196618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197302+0000","last_change":"2024-04-25T19:02:39.039435+0000","last_active":"2024-04-25T19:02:45.197302+0000","last_peered":"2024-04-25T19:02:45.197302+0000","last_clean":"2024-04-25T19:02:45.197302+0000","last_became_active":"2024-04-25T19:02:39.039084+0000","last_became_peered":"2024-04-25T19:02:39.039084+0000","last_unstale":"2024-04-25T19:02:45.197302+0000","last_undegraded":"2024-04-25T19:02:45.197302+0000","last_fullsized":"2024-04-25T19:02:45.197302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202918+0000","last_change":"2024-04-25T19:02:44.156161+0000","last_active":"2024-04-25T19:02:45.202918+0000","last_peered":"2024-04-25T19:02:45.202918+0000","last_clean":"2024-04-25T19:02:45.202918+0000","last_became_active":"2024-04-25T19:02:44.155666+0000","last_became_peered":"2024-04-25T19:02:44.155666+0000","last_unstale":"2024-04-25T19:02:45.202918+0000","last_undegraded":"2024-04-25T19:02:45.202918+0000","last_fullsized":"2024-04-25T19:02:45.202918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:22:00.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129573+0000","last_change":"2024-04-25T19:02:40.076702+0000","last_active":"2024-04-25T19:02:45.129573+0000","last_peered":"2024-04-25T19:02:45.129573+0000","last_clean":"2024-04-25T19:02:45.129573+0000","last_became_active":"2024-04-25T19:02:40.072663+0000","last_became_peered":"2024-04-25T19:02:40.072663+0000","last_unstale":"2024-04-25T19:02:45.129573+0000","last_undegraded":"2024-04-25T19:02:45.129573+0000","last_fullsized":"2024-04-25T19:02:45.129573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:35.926577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130496+0000","last_change":"2024-04-25T19:02:39.048307+0000","last_active":"2024-04-25T19:02:45.130496+0000","last_peered":"2024-04-25T19:02:45.130496+0000","last_clean":"2024-04-25T19:02:45.130496+0000","last_became_active":"2024-04-25T19:02:39.047902+0000","last_became_peered":"2024-04-25T19:02:39.047902+0000","last_unstale":"2024-04-25T19:02:45.130496+0000","last_undegraded":"2024-04-25T19:02:45.130496+0000","last_fullsized":"2024-04-25T19:02:45.130496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136907+0000","last_change":"2024-04-25T19:02:44.151849+0000","last_active":"2024-04-25T19:02:45.136907+0000","last_peered":"2024-04-25T19:02:45.136907+0000","last_clean":"2024-04-25T19:02:45.136907+0000","last_became_active":"2024-04-25T19:02:44.150781+0000","last_became_peered":"2024-04-25T19:02:44.150781+0000","last_unstale":"2024-04-25T19:02:45.136907+0000","last_undegraded":"2024-04-25T19:02:45.136907+0000","last_fullsized":"2024-04-25T19:02:45.136907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:30.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134288+0000","last_change":"2024-04-25T19:02:40.073292+0000","last_active":"2024-04-25T19:02:45.134288+0000","last_peered":"2024-04-25T19:02:45.134288+0000","last_clean":"2024-04-25T19:02:45.134288+0000","last_became_active":"2024-04-25T19:02:40.070890+0000","last_became_peered":"2024-04-25T19:02:40.070890+0000","last_unstale":"2024-04-25T19:02:45.134288+0000","last_undegraded":"2024-04-25T19:02:45.134288+0000","last_fullsized":"2024-04-25T19:02:45.134288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:19.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201740+0000","last_change":"2024-04-25T19:02:39.053287+0000","last_active":"2024-04-25T19:02:45.201740+0000","last_peered":"2024-04-25T19:02:45.201740+0000","last_clean":"2024-04-25T19:02:45.201740+0000","last_became_active":"2024-04-25T19:02:39.052037+0000","last_became_peered":"2024-04-25T19:02:39.052037+0000","last_unstale":"2024-04-25T19:02:45.201740+0000","last_undegraded":"2024-04-25T19:02:45.201740+0000","last_fullsized":"2024-04-25T19:02:45.201740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130560+0000","last_change":"2024-04-25T19:02:44.158840+0000","last_active":"2024-04-25T19:02:45.130560+0000","last_peered":"2024-04-25T19:02:45.130560+0000","last_clean":"2024-04-25T19:02:45.130560+0000","last_became_active":"2024-04-25T19:02:44.156508+0000","last_became_peered":"2024-04-25T19:02:44.156508+0000","last_unstale":"2024-04-25T19:02:45.130560+0000","last_undegraded":"2024-04-25T19:02:45.130560+0000","last_fullsized":"2024-04-25T19:02:45.130560+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:08.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198990+0000","last_change":"2024-04-25T19:02:40.084277+0000","last_active":"2024-04-25T19:02:45.198990+0000","last_peered":"2024-04-25T19:02:45.198990+0000","last_clean":"2024-04-25T19:02:45.198990+0000","last_became_active":"2024-04-25T19:02:40.083896+0000","last_became_peered":"2024-04-25T19:02:40.083896+0000","last_unstale":"2024-04-25T19:02:45.198990+0000","last_undegraded":"2024-04-25T19:02:45.198990+0000","last_fullsized":"2024-04-25T19:02:45.198990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.393874+0000","last_change":"2024-04-25T19:02:39.045230+0000","last_active":"2024-04-25T19:03:00.393874+0000","last_peered":"2024-04-25T19:03:00.393874+0000","last_clean":"2024-04-25T19:03:00.393874+0000","last_became_active":"2024-04-25T19:02:39.044884+0000","last_became_peered":"2024-04-25T19:02:39.044884+0000","last_unstale":"2024-04-25T19:03:00.393874+0000","last_undegraded":"2024-04-25T19:03:00.393874+0000","last_fullsized":"2024-04-25T19:03:00.393874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:20.905809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133971+0000","last_change":"2024-04-25T19:02:44.153433+0000","last_active":"2024-04-25T19:02:45.133971+0000","last_peered":"2024-04-25T19:02:45.133971+0000","last_clean":"2024-04-25T19:02:45.133971+0000","last_became_active":"2024-04-25T19:02:44.152935+0000","last_became_peered":"2024-04-25T19:02:44.152935+0000","last_unstale":"2024-04-25T19:02:45.133971+0000","last_undegraded":"2024-04-25T19:02:45.133971+0000","last_fullsized":"2024-04-25T19:02:45.133971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:35.591324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129614+0000","last_change":"2024-04-25T19:02:40.074340+0000","last_active":"2024-04-25T19:02:45.129614+0000","last_peered":"2024-04-25T19:02:45.129614+0000","last_clean":"2024-04-25T19:02:45.129614+0000","last_became_active":"2024-04-25T19:02:40.069150+0000","last_became_peered":"2024-04-25T19:02:40.069150+0000","last_unstale":"2024-04-25T19:02:45.129614+0000","last_undegraded":"2024-04-25T19:02:45.129614+0000","last_fullsized":"2024-04-25T19:02:45.129614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:53.328375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203162+0000","last_change":"2024-04-25T19:02:39.050352+0000","last_active":"2024-04-25T19:02:45.203162+0000","last_peered":"2024-04-25T19:02:45.203162+0000","last_clean":"2024-04-25T19:02:45.203162+0000","last_became_active":"2024-04-25T19:02:39.049837+0000","last_became_peered":"2024-04-25T19:02:39.049837+0000","last_unstale":"2024-04-25T19:02:45.203162+0000","last_undegraded":"2024-04-25T19:02:45.203162+0000","last_fullsized":"2024-04-25T19:02:45.203162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:46.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200045+0000","last_change":"2024-04-25T19:02:44.147062+0000","last_active":"2024-04-25T19:02:45.200045+0000","last_peered":"2024-04-25T19:02:45.200045+0000","last_clean":"2024-04-25T19:02:45.200045+0000","last_became_active":"2024-04-25T19:02:44.145997+0000","last_became_peered":"2024-04-25T19:02:44.145997+0000","last_unstale":"2024-04-25T19:02:45.200045+0000","last_undegraded":"2024-04-25T19:02:45.200045+0000","last_fullsized":"2024-04-25T19:02:45.200045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203397+0000","last_change":"2024-04-25T19:02:40.083756+0000","last_active":"2024-04-25T19:02:45.203397+0000","last_peered":"2024-04-25T19:02:45.203397+0000","last_clean":"2024-04-25T19:02:45.203397+0000","last_became_active":"2024-04-25T19:02:40.083402+0000","last_became_peered":"2024-04-25T19:02:40.083402+0000","last_unstale":"2024-04-25T19:02:45.203397+0000","last_undegraded":"2024-04-25T19:02:45.203397+0000","last_fullsized":"2024-04-25T19:02:45.203397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198142+0000","last_change":"2024-04-25T19:02:39.043121+0000","last_active":"2024-04-25T19:02:45.198142+0000","last_peered":"2024-04-25T19:02:45.198142+0000","last_clean":"2024-04-25T19:02:45.198142+0000","last_became_active":"2024-04-25T19:02:39.042778+0000","last_became_peered":"2024-04-25T19:02:39.042778+0000","last_unstale":"2024-04-25T19:02:45.198142+0000","last_undegraded":"2024-04-25T19:02:45.198142+0000","last_fullsized":"2024-04-25T19:02:45.198142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199456+0000","last_change":"2024-04-25T19:02:44.148628+0000","last_active":"2024-04-25T19:02:45.199456+0000","last_peered":"2024-04-25T19:02:45.199456+0000","last_clean":"2024-04-25T19:02:45.199456+0000","last_became_active":"2024-04-25T19:02:44.147929+0000","last_became_peered":"2024-04-25T19:02:44.147929+0000","last_unstale":"2024-04-25T19:02:45.199456+0000","last_undegraded":"2024-04-25T19:02:45.199456+0000","last_fullsized":"2024-04-25T19:02:45.199456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199371+0000","last_change":"2024-04-25T19:02:40.069980+0000","last_active":"2024-04-25T19:02:45.199371+0000","last_peered":"2024-04-25T19:02:45.199371+0000","last_clean":"2024-04-25T19:02:45.199371+0000","last_became_active":"2024-04-25T19:02:40.069323+0000","last_became_peered":"2024-04-25T19:02:40.069323+0000","last_unstale":"2024-04-25T19:02:45.199371+0000","last_undegraded":"2024-04-25T19:02:45.199371+0000","last_fullsized":"2024-04-25T19:02:45.199371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200662+0000","last_change":"2024-04-25T19:02:39.044027+0000","last_active":"2024-04-25T19:02:45.200662+0000","last_peered":"2024-04-25T19:02:45.200662+0000","last_clean":"2024-04-25T19:02:45.200662+0000","last_became_active":"2024-04-25T19:02:39.043696+0000","last_became_peered":"2024-04-25T19:02:39.043696+0000","last_unstale":"2024-04-25T19:02:45.200662+0000","last_undegraded":"2024-04-25T19:02:45.200662+0000","last_fullsized":"2024-04-25T19:02:45.200662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132192+0000","last_change":"2024-04-25T19:02:44.147513+0000","last_active":"2024-04-25T19:02:45.132192+0000","last_peered":"2024-04-25T19:02:45.132192+0000","last_clean":"2024-04-25T19:02:45.132192+0000","last_became_active":"2024-04-25T19:02:44.146937+0000","last_became_peered":"2024-04-25T19:02:44.146937+0000","last_unstale":"2024-04-25T19:02:45.132192+0000","last_undegraded":"2024-04-25T19:02:45.132192+0000","last_fullsized":"2024-04-25T19:02:45.132192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135996+0000","last_change":"2024-04-25T19:02:44.153340+0000","last_active":"2024-04-25T19:02:45.135996+0000","last_peered":"2024-04-25T19:02:45.135996+0000","last_clean":"2024-04-25T19:02:45.135996+0000","last_became_active":"2024-04-25T19:02:44.152850+0000","last_became_peered":"2024-04-25T19:02:44.152850+0000","last_unstale":"2024-04-25T19:02:45.135996+0000","last_undegraded":"2024-04-25T19:02:45.135996+0000","last_fullsized":"2024-04-25T19:02:45.135996+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:57.414588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132085+0000","last_change":"2024-04-25T19:02:40.051844+0000","last_active":"2024-04-25T19:02:45.132085+0000","last_peered":"2024-04-25T19:02:45.132085+0000","last_clean":"2024-04-25T19:02:45.132085+0000","last_became_active":"2024-04-25T19:02:40.050922+0000","last_became_peered":"2024-04-25T19:02:40.050922+0000","last_unstale":"2024-04-25T19:02:45.132085+0000","last_undegraded":"2024-04-25T19:02:45.132085+0000","last_fullsized":"2024-04-25T19:02:45.132085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.344370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129179+0000","last_change":"2024-04-25T19:02:39.045270+0000","last_active":"2024-04-25T19:02:45.129179+0000","last_peered":"2024-04-25T19:02:45.129179+0000","last_clean":"2024-04-25T19:02:45.129179+0000","last_became_active":"2024-04-25T19:02:39.044936+0000","last_became_peered":"2024-04-25T19:02:39.044936+0000","last_unstale":"2024-04-25T19:02:45.129179+0000","last_undegraded":"2024-04-25T19:02:45.129179+0000","last_fullsized":"2024-04-25T19:02:45.129179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131178+0000","last_change":"2024-04-25T19:02:44.160881+0000","last_active":"2024-04-25T19:02:45.131178+0000","last_peered":"2024-04-25T19:02:45.131178+0000","last_clean":"2024-04-25T19:02:45.131178+0000","last_became_active":"2024-04-25T19:02:44.160267+0000","last_became_peered":"2024-04-25T19:02:44.160267+0000","last_unstale":"2024-04-25T19:02:45.131178+0000","last_undegraded":"2024-04-25T19:02:45.131178+0000","last_fullsized":"2024-04-25T19:02:45.131178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:21.942974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199361+0000","last_change":"2024-04-25T19:02:40.065898+0000","last_active":"2024-04-25T19:02:45.199361+0000","last_peered":"2024-04-25T19:02:45.199361+0000","last_clean":"2024-04-25T19:02:45.199361+0000","last_became_active":"2024-04-25T19:02:40.062397+0000","last_became_peered":"2024-04-25T19:02:40.062397+0000","last_unstale":"2024-04-25T19:02:45.199361+0000","last_undegraded":"2024-04-25T19:02:45.199361+0000","last_fullsized":"2024-04-25T19:02:45.199361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132878+0000","last_change":"2024-04-25T19:02:39.049593+0000","last_active":"2024-04-25T19:02:45.132878+0000","last_peered":"2024-04-25T19:02:45.132878+0000","last_clean":"2024-04-25T19:02:45.132878+0000","last_became_active":"2024-04-25T19:02:39.049140+0000","last_became_peered":"2024-04-25T19:02:39.049140+0000","last_unstale":"2024-04-25T19:02:45.132878+0000","last_undegraded":"2024-04-25T19:02:45.132878+0000","last_fullsized":"2024-04-25T19:02:45.132878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:01.624014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130188+0000","last_change":"2024-04-25T19:02:39.050611+0000","last_active":"2024-04-25T19:02:45.130188+0000","last_peered":"2024-04-25T19:02:45.130188+0000","last_clean":"2024-04-25T19:02:45.130188+0000","last_became_active":"2024-04-25T19:02:39.050319+0000","last_became_peered":"2024-04-25T19:02:39.050319+0000","last_unstale":"2024-04-25T19:02:45.130188+0000","last_undegraded":"2024-04-25T19:02:45.130188+0000","last_fullsized":"2024-04-25T19:02:45.130188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132040+0000","last_change":"2024-04-25T19:02:40.076010+0000","last_active":"2024-04-25T19:02:45.132040+0000","last_peered":"2024-04-25T19:02:45.132040+0000","last_clean":"2024-04-25T19:02:45.132040+0000","last_became_active":"2024-04-25T19:02:40.075677+0000","last_became_peered":"2024-04-25T19:02:40.075677+0000","last_unstale":"2024-04-25T19:02:45.132040+0000","last_undegraded":"2024-04-25T19:02:45.132040+0000","last_fullsized":"2024-04-25T19:02:45.132040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135539+0000","last_change":"2024-04-25T19:02:44.154362+0000","last_active":"2024-04-25T19:02:45.135539+0000","last_peered":"2024-04-25T19:02:45.135539+0000","last_clean":"2024-04-25T19:02:45.135539+0000","last_became_active":"2024-04-25T19:02:44.153509+0000","last_became_peered":"2024-04-25T19:02:44.153509+0000","last_unstale":"2024-04-25T19:02:45.135539+0000","last_undegraded":"2024-04-25T19:02:45.135539+0000","last_fullsized":"2024-04-25T19:02:45.135539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:16.467033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201054+0000","last_change":"2024-04-25T19:02:39.045741+0000","last_active":"2024-04-25T19:02:45.201054+0000","last_peered":"2024-04-25T19:02:45.201054+0000","last_clean":"2024-04-25T19:02:45.201054+0000","last_became_active":"2024-04-25T19:02:39.045015+0000","last_became_peered":"2024-04-25T19:02:39.045015+0000","last_unstale":"2024-04-25T19:02:45.201054+0000","last_undegraded":"2024-04-25T19:02:45.201054+0000","last_fullsized":"2024-04-25T19:02:45.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199483+0000","last_change":"2024-04-25T19:02:40.080804+0000","last_active":"2024-04-25T19:02:45.199483+0000","last_peered":"2024-04-25T19:02:45.199483+0000","last_clean":"2024-04-25T19:02:45.199483+0000","last_became_active":"2024-04-25T19:02:40.080421+0000","last_became_peered":"2024-04-25T19:02:40.080421+0000","last_unstale":"2024-04-25T19:02:45.199483+0000","last_undegraded":"2024-04-25T19:02:45.199483+0000","last_fullsized":"2024-04-25T19:02:45.199483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202500+0000","last_change":"2024-04-25T19:02:44.163067+0000","last_active":"2024-04-25T19:02:45.202500+0000","last_peered":"2024-04-25T19:02:45.202500+0000","last_clean":"2024-04-25T19:02:45.202500+0000","last_became_active":"2024-04-25T19:02:44.162000+0000","last_became_peered":"2024-04-25T19:02:44.162000+0000","last_unstale":"2024-04-25T19:02:45.202500+0000","last_undegraded":"2024-04-25T19:02:45.202500+0000","last_fullsized":"2024-04-25T19:02:45.202500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:59.430583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198599+0000","last_change":"2024-04-25T19:02:40.063088+0000","last_active":"2024-04-25T19:02:45.198599+0000","last_peered":"2024-04-25T19:02:45.198599+0000","last_clean":"2024-04-25T19:02:45.198599+0000","last_became_active":"2024-04-25T19:02:40.062379+0000","last_became_peered":"2024-04-25T19:02:40.062379+0000","last_unstale":"2024-04-25T19:02:45.198599+0000","last_undegraded":"2024-04-25T19:02:45.198599+0000","last_fullsized":"2024-04-25T19:02:45.198599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200902+0000","last_change":"2024-04-25T19:02:40.065240+0000","last_active":"2024-04-25T19:02:45.200902+0000","last_peered":"2024-04-25T19:02:45.200902+0000","last_clean":"2024-04-25T19:02:45.200902+0000","last_became_active":"2024-04-25T19:02:40.064900+0000","last_became_peered":"2024-04-25T19:02:40.064900+0000","last_unstale":"2024-04-25T19:02:45.200902+0000","last_undegraded":"2024-04-25T19:02:45.200902+0000","last_fullsized":"2024-04-25T19:02:45.200902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:34.156431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129604+0000","last_change":"2024-04-25T19:02:40.084316+0000","last_active":"2024-04-25T19:02:45.129604+0000","last_peered":"2024-04-25T19:02:45.129604+0000","last_clean":"2024-04-25T19:02:45.129604+0000","last_became_active":"2024-04-25T19:02:40.084014+0000","last_became_peered":"2024-04-25T19:02:40.084014+0000","last_unstale":"2024-04-25T19:02:45.129604+0000","last_undegraded":"2024-04-25T19:02:45.129604+0000","last_fullsized":"2024-04-25T19:02:45.129604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200778+0000","last_change":"2024-04-25T19:02:40.070357+0000","last_active":"2024-04-25T19:02:45.200778+0000","last_peered":"2024-04-25T19:02:45.200778+0000","last_clean":"2024-04-25T19:02:45.200778+0000","last_became_active":"2024-04-25T19:02:40.070028+0000","last_became_peered":"2024-04-25T19:02:40.070028+0000","last_unstale":"2024-04-25T19:02:45.200778+0000","last_undegraded":"2024-04-25T19:02:45.200778+0000","last_fullsized":"2024-04-25T19:02:45.200778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137153+0000","last_change":"2024-04-25T19:02:40.088098+0000","last_active":"2024-04-25T19:02:45.137153+0000","last_peered":"2024-04-25T19:02:45.137153+0000","last_clean":"2024-04-25T19:02:45.137153+0000","last_became_active":"2024-04-25T19:02:40.087738+0000","last_became_peered":"2024-04-25T19:02:40.087738+0000","last_unstale":"2024-04-25T19:02:45.137153+0000","last_undegraded":"2024-04-25T19:02:45.137153+0000","last_fullsized":"2024-04-25T19:02:45.137153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:08.428103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137734+0000","last_change":"2024-04-25T19:02:40.063201+0000","last_active":"2024-04-25T19:02:45.137734+0000","last_peered":"2024-04-25T19:02:45.137734+0000","last_clean":"2024-04-25T19:02:45.137734+0000","last_became_active":"2024-04-25T19:02:40.060739+0000","last_became_peered":"2024-04-25T19:02:40.060739+0000","last_unstale":"2024-04-25T19:02:45.137734+0000","last_undegraded":"2024-04-25T19:02:45.137734+0000","last_fullsized":"2024-04-25T19:02:45.137734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:40.756295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199236+0000","last_change":"2024-04-25T19:02:40.076099+0000","last_active":"2024-04-25T19:02:45.199236+0000","last_peered":"2024-04-25T19:02:45.199236+0000","last_clean":"2024-04-25T19:02:45.199236+0000","last_became_active":"2024-04-25T19:02:40.075690+0000","last_became_peered":"2024-04-25T19:02:40.075690+0000","last_unstale":"2024-04-25T19:02:45.199236+0000","last_undegraded":"2024-04-25T19:02:45.199236+0000","last_fullsized":"2024-04-25T19:02:45.199236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202228+0000","last_change":"2024-04-25T19:02:40.075377+0000","last_active":"2024-04-25T19:02:45.202228+0000","last_peered":"2024-04-25T19:02:45.202228+0000","last_clean":"2024-04-25T19:02:45.202228+0000","last_became_active":"2024-04-25T19:02:40.069488+0000","last_became_peered":"2024-04-25T19:02:40.069488+0000","last_unstale":"2024-04-25T19:02:45.202228+0000","last_undegraded":"2024-04-25T19:02:45.202228+0000","last_fullsized":"2024-04-25T19:02:45.202228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128848+0000","last_change":"2024-04-25T19:02:40.083952+0000","last_active":"2024-04-25T19:02:45.128848+0000","last_peered":"2024-04-25T19:02:45.128848+0000","last_clean":"2024-04-25T19:02:45.128848+0000","last_became_active":"2024-04-25T19:02:40.083612+0000","last_became_peered":"2024-04-25T19:02:40.083612+0000","last_unstale":"2024-04-25T19:02:45.128848+0000","last_undegraded":"2024-04-25T19:02:45.128848+0000","last_fullsized":"2024-04-25T19:02:45.128848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199584+0000","last_change":"2024-04-25T19:02:40.067612+0000","last_active":"2024-04-25T19:02:45.199584+0000","last_peered":"2024-04-25T19:02:45.199584+0000","last_clean":"2024-04-25T19:02:45.199584+0000","last_became_active":"2024-04-25T19:02:40.067294+0000","last_became_peered":"2024-04-25T19:02:40.067294+0000","last_unstale":"2024-04-25T19:02:45.199584+0000","last_undegraded":"2024-04-25T19:02:45.199584+0000","last_fullsized":"2024-04-25T19:02:45.199584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197462+0000","last_change":"2024-04-25T19:02:40.073442+0000","last_active":"2024-04-25T19:02:45.197462+0000","last_peered":"2024-04-25T19:02:45.197462+0000","last_clean":"2024-04-25T19:02:45.197462+0000","last_became_active":"2024-04-25T19:02:40.073045+0000","last_became_peered":"2024-04-25T19:02:40.073045+0000","last_unstale":"2024-04-25T19:02:45.197462+0000","last_undegraded":"2024-04-25T19:02:45.197462+0000","last_fullsized":"2024-04-25T19:02:45.197462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131761+0000","last_change":"2024-04-25T19:02:40.050494+0000","last_active":"2024-04-25T19:02:45.131761+0000","last_peered":"2024-04-25T19:02:45.131761+0000","last_clean":"2024-04-25T19:02:45.131761+0000","last_became_active":"2024-04-25T19:02:40.050067+0000","last_became_peered":"2024-04-25T19:02:40.050067+0000","last_unstale":"2024-04-25T19:02:45.131761+0000","last_undegraded":"2024-04-25T19:02:45.131761+0000","last_fullsized":"2024-04-25T19:02:45.131761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198872+0000","last_change":"2024-04-25T19:02:40.071347+0000","last_active":"2024-04-25T19:02:45.198872+0000","last_peered":"2024-04-25T19:02:45.198872+0000","last_clean":"2024-04-25T19:02:45.198872+0000","last_became_active":"2024-04-25T19:02:40.071062+0000","last_became_peered":"2024-04-25T19:02:40.071062+0000","last_unstale":"2024-04-25T19:02:45.198872+0000","last_undegraded":"2024-04-25T19:02:45.198872+0000","last_fullsized":"2024-04-25T19:02:45.198872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130821+0000","last_change":"2024-04-25T19:02:40.076710+0000","last_active":"2024-04-25T19:02:45.130821+0000","last_peered":"2024-04-25T19:02:45.130821+0000","last_clean":"2024-04-25T19:02:45.130821+0000","last_became_active":"2024-04-25T19:02:40.071132+0000","last_became_peered":"2024-04-25T19:02:40.071132+0000","last_unstale":"2024-04-25T19:02:45.130821+0000","last_undegraded":"2024-04-25T19:02:45.130821+0000","last_fullsized":"2024-04-25T19:02:45.130821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:21:56.124743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202755+0000","last_change":"2024-04-25T19:02:40.076746+0000","last_active":"2024-04-25T19:02:45.202755+0000","last_peered":"2024-04-25T19:02:45.202755+0000","last_clean":"2024-04-25T19:02:45.202755+0000","last_became_active":"2024-04-25T19:02:40.068835+0000","last_became_peered":"2024-04-25T19:02:40.068835+0000","last_unstale":"2024-04-25T19:02:45.202755+0000","last_undegraded":"2024-04-25T19:02:45.202755+0000","last_fullsized":"2024-04-25T19:02:45.202755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132119+0000","last_change":"2024-04-25T19:02:40.090029+0000","last_active":"2024-04-25T19:02:45.132119+0000","last_peered":"2024-04-25T19:02:45.132119+0000","last_clean":"2024-04-25T19:02:45.132119+0000","last_became_active":"2024-04-25T19:02:40.088886+0000","last_became_peered":"2024-04-25T19:02:40.088886+0000","last_unstale":"2024-04-25T19:02:45.132119+0000","last_undegraded":"2024-04-25T19:02:45.132119+0000","last_fullsized":"2024-04-25T19:02:45.132119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:38.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128770+0000","last_change":"2024-04-25T19:02:40.079792+0000","last_active":"2024-04-25T19:02:45.128770+0000","last_peered":"2024-04-25T19:02:45.128770+0000","last_clean":"2024-04-25T19:02:45.128770+0000","last_became_active":"2024-04-25T19:02:40.071455+0000","last_became_peered":"2024-04-25T19:02:40.071455+0000","last_unstale":"2024-04-25T19:02:45.128770+0000","last_undegraded":"2024-04-25T19:02:45.128770+0000","last_fullsized":"2024-04-25T19:02:45.128770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198995+0000","last_change":"2024-04-25T19:02:40.088240+0000","last_active":"2024-04-25T19:02:45.198995+0000","last_peered":"2024-04-25T19:02:45.198995+0000","last_clean":"2024-04-25T19:02:45.198995+0000","last_became_active":"2024-04-25T19:02:40.087892+0000","last_became_peered":"2024-04-25T19:02:40.087892+0000","last_unstale":"2024-04-25T19:02:45.198995+0000","last_undegraded":"2024-04-25T19:02:45.198995+0000","last_fullsized":"2024-04-25T19:02:45.198995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133970+0000","last_change":"2024-04-25T19:02:40.062612+0000","last_active":"2024-04-25T19:02:45.133970+0000","last_peered":"2024-04-25T19:02:45.133970+0000","last_clean":"2024-04-25T19:02:45.133970+0000","last_became_active":"2024-04-25T19:02:40.060475+0000","last_became_peered":"2024-04-25T19:02:40.060475+0000","last_unstale":"2024-04-25T19:02:45.133970+0000","last_undegraded":"2024-04-25T19:02:45.133970+0000","last_fullsized":"2024-04-25T19:02:45.133970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:10.198621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135782+0000","last_change":"2024-04-25T19:02:40.091050+0000","last_active":"2024-04-25T19:02:45.135782+0000","last_peered":"2024-04-25T19:02:45.135782+0000","last_clean":"2024-04-25T19:02:45.135782+0000","last_became_active":"2024-04-25T19:02:40.090640+0000","last_became_peered":"2024-04-25T19:02:40.090640+0000","last_unstale":"2024-04-25T19:02:45.135782+0000","last_undegraded":"2024-04-25T19:02:45.135782+0000","last_fullsized":"2024-04-25T19:02:45.135782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:57.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198750+0000","last_change":"2024-04-25T19:02:40.074894+0000","last_active":"2024-04-25T19:02:45.198750+0000","last_peered":"2024-04-25T19:02:45.198750+0000","last_clean":"2024-04-25T19:02:45.198750+0000","last_became_active":"2024-04-25T19:02:40.074142+0000","last_became_peered":"2024-04-25T19:02:40.074142+0000","last_unstale":"2024-04-25T19:02:45.198750+0000","last_undegraded":"2024-04-25T19:02:45.198750+0000","last_fullsized":"2024-04-25T19:02:45.198750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198798+0000","last_change":"2024-04-25T19:02:40.087831+0000","last_active":"2024-04-25T19:02:45.198798+0000","last_peered":"2024-04-25T19:02:45.198798+0000","last_clean":"2024-04-25T19:02:45.198798+0000","last_became_active":"2024-04-25T19:02:40.087495+0000","last_became_peered":"2024-04-25T19:02:40.087495+0000","last_unstale":"2024-04-25T19:02:45.198798+0000","last_undegraded":"2024-04-25T19:02:45.198798+0000","last_fullsized":"2024-04-25T19:02:45.198798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197993+0000","last_change":"2024-04-25T19:02:40.067092+0000","last_active":"2024-04-25T19:02:45.197993+0000","last_peered":"2024-04-25T19:02:45.197993+0000","last_clean":"2024-04-25T19:02:45.197993+0000","last_became_active":"2024-04-25T19:02:40.066752+0000","last_became_peered":"2024-04-25T19:02:40.066752+0000","last_unstale":"2024-04-25T19:02:45.197993+0000","last_undegraded":"2024-04-25T19:02:45.197993+0000","last_fullsized":"2024-04-25T19:02:45.197993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200560+0000","last_change":"2024-04-25T19:02:40.069596+0000","last_active":"2024-04-25T19:02:45.200560+0000","last_peered":"2024-04-25T19:02:45.200560+0000","last_clean":"2024-04-25T19:02:45.200560+0000","last_became_active":"2024-04-25T19:02:40.069272+0000","last_became_peered":"2024-04-25T19:02:40.069272+0000","last_unstale":"2024-04-25T19:02:45.200560+0000","last_undegraded":"2024-04-25T19:02:45.200560+0000","last_fullsized":"2024-04-25T19:02:45.200560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198925+0000","last_change":"2024-04-25T19:02:40.085542+0000","last_active":"2024-04-25T19:02:45.198925+0000","last_peered":"2024-04-25T19:02:45.198925+0000","last_clean":"2024-04-25T19:02:45.198925+0000","last_became_active":"2024-04-25T19:02:40.085203+0000","last_became_peered":"2024-04-25T19:02:40.085203+0000","last_unstale":"2024-04-25T19:02:45.198925+0000","last_undegraded":"2024-04-25T19:02:45.198925+0000","last_fullsized":"2024-04-25T19:02:45.198925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197180+0000","last_change":"2024-04-25T19:02:40.070062+0000","last_active":"2024-04-25T19:02:45.197180+0000","last_peered":"2024-04-25T19:02:45.197180+0000","last_clean":"2024-04-25T19:02:45.197180+0000","last_became_active":"2024-04-25T19:02:40.069224+0000","last_became_peered":"2024-04-25T19:02:40.069224+0000","last_unstale":"2024-04-25T19:02:45.197180+0000","last_undegraded":"2024-04-25T19:02:45.197180+0000","last_fullsized":"2024-04-25T19:02:45.197180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198254+0000","last_change":"2024-04-25T19:02:40.055714+0000","last_active":"2024-04-25T19:02:45.198254+0000","last_peered":"2024-04-25T19:02:45.198254+0000","last_clean":"2024-04-25T19:02:45.198254+0000","last_became_active":"2024-04-25T19:02:40.052418+0000","last_became_peered":"2024-04-25T19:02:40.052418+0000","last_unstale":"2024-04-25T19:02:45.198254+0000","last_undegraded":"2024-04-25T19:02:45.198254+0000","last_fullsized":"2024-04-25T19:02:45.198254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198608+0000","last_change":"2024-04-25T19:02:40.087716+0000","last_active":"2024-04-25T19:02:45.198608+0000","last_peered":"2024-04-25T19:02:45.198608+0000","last_clean":"2024-04-25T19:02:45.198608+0000","last_became_active":"2024-04-25T19:02:40.085461+0000","last_became_peered":"2024-04-25T19:02:40.085461+0000","last_unstale":"2024-04-25T19:02:45.198608+0000","last_undegraded":"2024-04-25T19:02:45.198608+0000","last_fullsized":"2024-04-25T19:02:45.198608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198153+0000","last_change":"2024-04-25T19:02:40.053676+0000","last_active":"2024-04-25T19:02:45.198153+0000","last_peered":"2024-04-25T19:02:45.198153+0000","last_clean":"2024-04-25T19:02:45.198153+0000","last_became_active":"2024-04-25T19:02:40.052518+0000","last_became_peered":"2024-04-25T19:02:40.052518+0000","last_unstale":"2024-04-25T19:02:45.198153+0000","last_undegraded":"2024-04-25T19:02:45.198153+0000","last_fullsized":"2024-04-25T19:02:45.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:54.192825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201739+0000","last_change":"2024-04-25T19:02:40.076362+0000","last_active":"2024-04-25T19:02:45.201739+0000","last_peered":"2024-04-25T19:02:45.201739+0000","last_clean":"2024-04-25T19:02:45.201739+0000","last_became_active":"2024-04-25T19:02:40.070187+0000","last_became_peered":"2024-04-25T19:02:40.070187+0000","last_unstale":"2024-04-25T19:02:45.201739+0000","last_undegraded":"2024-04-25T19:02:45.201739+0000","last_fullsized":"2024-04-25T19:02:45.201739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202327+0000","last_change":"2024-04-25T19:02:40.072359+0000","last_active":"2024-04-25T19:02:45.202327+0000","last_peered":"2024-04-25T19:02:45.202327+0000","last_clean":"2024-04-25T19:02:45.202327+0000","last_became_active":"2024-04-25T19:02:40.071678+0000","last_became_peered":"2024-04-25T19:02:40.071678+0000","last_unstale":"2024-04-25T19:02:45.202327+0000","last_undegraded":"2024-04-25T19:02:45.202327+0000","last_fullsized":"2024-04-25T19:02:45.202327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198720+0000","last_change":"2024-04-25T19:02:40.078568+0000","last_active":"2024-04-25T19:02:45.198720+0000","last_peered":"2024-04-25T19:02:45.198720+0000","last_clean":"2024-04-25T19:02:45.198720+0000","last_became_active":"2024-04-25T19:02:40.078287+0000","last_became_peered":"2024-04-25T19:02:40.078287+0000","last_unstale":"2024-04-25T19:02:45.198720+0000","last_undegraded":"2024-04-25T19:02:45.198720+0000","last_fullsized":"2024-04-25T19:02:45.198720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:05.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:03:05.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.644+0000 7f2ea3200640 1 Processor -- start 2024-04-25T19:03:05.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.644+0000 7f2ea3200640 1 -- start start 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2ea406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea3200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ea406dac0 0x7f2ea4142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea406ded0 con 0x7f2ea4142a70 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea3200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea40b5650 con 0x7f2ea406dac0 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea40b5c70 con 0x7f2ea413fbf0 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea2800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2ea4142a70 0x7f2ea41607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45236/0 (socket says 172.21.15.123:45236) 2024-04-25T19:03:05.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.646+0000 7f2ea2800640 1 -- 172.21.15.123:0/2206583435 learned_addr learned my addr 172.21.15.123:0/2206583435 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:05.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2e9be00640 1 --2- 172.21.15.123:0/2206583435 >> v2:172.21.15.146:3300/0 conn(0x7f2ea406dac0 0x7f2ea4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1e00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2ea406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330282966 0 0) 0x7f2e9c001490 con 0x7f2ea4142a70 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e88005730 con 0x7f2ea4142a70 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 126642824 0 0) 0x7f2e88005730 con 0x7f2ea4142a70 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 msgr2=0x7f2ea406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2ea406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 >> v2:172.21.15.146:3300/0 conn(0x7f2ea406dac0 msgr2=0x7f2ea4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 --2- 172.21.15.123:0/2206583435 >> v2:172.21.15.146:3300/0 conn(0x7f2ea406dac0 0x7f2ea4142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea40b5f50 con 0x7f2ea4142a70 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f2e9c001bc0 con 0x7f2ea4142a70 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 >> v1:172.21.15.123:6789/0 conn(0x7f2ea4142a70 legacy=0x7f2ea41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88005bc0 0x7f2e88006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.647+0000 7f2ea1400640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e88006a10 0x7f2e88006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea1400640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2e88007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b5650 con 0x7f2e88006a10 2024-04-25T19:03:05.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b5c70 con 0x7f2e88005bc0 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8800dc00 con 0x7f2ea413fbf0 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e88006a10 0x7f2e88006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea1e00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88005bc0 0x7f2e88006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2ea2800640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2e88007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 -- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 msgr2=0x7f2e88007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ea413fbf0 0x7f2e88007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 -- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88005bc0 msgr2=0x7f2e88006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88005bc0 0x7f2e88006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.648+0000 7f2e9be00640 1 -- 172.21.15.123:0/2206583435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e88005730 con 0x7f2e88006a10 2024-04-25T19:03:05.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.649+0000 7f2e9be00640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e88006a10 0x7f2e88006e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009ac0 tx=0x7f2e88007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db8aa039661914c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c036070 con 0x7f2e88006a10 2024-04-25T19:03:05.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8c031400 con 0x7f2e88006a10 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea1400640 1 -- 172.21.15.123:0/2206583435 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c009690 con 0x7f2e88006a10 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea3200640 1 -- 172.21.15.123:0/2206583435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e88006a10 msgr2=0x7f2e88006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea3200640 1 --2- 172.21.15.123:0/2206583435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e88006a10 0x7f2e88006e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009ac0 tx=0x7f2e88007a40 comp rx=0 tx=0).stop 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea2800640 1 -- 172.21.15.123:0/2206583435 reap_dead start 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea3200640 1 -- 172.21.15.123:0/2206583435 shutdown_connections 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.650+0000 7f2ea3200640 1 -- 172.21.15.123:0/2206583435 >> 172.21.15.123:0/2206583435 conn(0x7f2ea4073200 msgr2=0x7f2ea4073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.651+0000 7f2ea3200640 1 -- 172.21.15.123:0/2206583435 shutdown_connections 2024-04-25T19:03:05.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.651+0000 7f2ea3200640 1 -- 172.21.15.123:0/2206583435 wait complete. 2024-04-25T19:03:05.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.653+0000 7f2ea3200640 1 Processor -- start 2024-04-25T19:03:05.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.653+0000 7f2ea3200640 1 -- start start 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88006a10 0x7f2ea4156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ea41f9bc0 0x7f2ea41f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b6a30 con 0x7f2ea41fa510 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b6510 con 0x7f2ea41f9bc0 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea4142e80 con 0x7f2e88006a10 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34124/0 (socket says 172.21.15.123:34124) 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 -- 172.21.15.123:0/3956067577 learned_addr learned my addr 172.21.15.123:0/3956067577 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:05.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea1e00640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88006a10 0x7f2ea4156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 -- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88006a10 msgr2=0x7f2ea4156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88006a10 0x7f2ea4156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 -- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ea41f9bc0 msgr2=0x7f2ea41f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ea41f9bc0 0x7f2ea41f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.655+0000 7f2ea2800640 1 -- 172.21.15.123:0/3956067577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea415b3c0 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2ea2800640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ea415d1b0 tx=0x7f2e9c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00eda0 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c018070 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea413e770 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c013660 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.656+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea415b940 con 0x7f2ea41fa510 2024-04-25T19:03:05.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.657+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea413f3a0 con 0x7f2ea41fa510 2024-04-25T19:03:05.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.659+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2e9c013800 con 0x7f2ea41fa510 2024-04-25T19:03:05.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.659+0000 7f2e9a000640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 0x7f2e800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:05.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.660+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e9c095a00 con 0x7f2ea41fa510 2024-04-25T19:03:05.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.660+0000 7f2ea1e00640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 0x7f2e800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.660+0000 7f2ea1e00640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 0x7f2e800769f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009160 tx=0x7f2e94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.664+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ea406ded0 con 0x7f2ea41fa510 2024-04-25T19:03:05.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.863+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ea4050740 con 0x7f2e800744e0 2024-04-25T19:03:05.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.887+0000 7f2e9a000640 1 -- 172.21.15.123:0/3956067577 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332559 (secure 0 0 0) 0x7f2ea4050740 con 0x7f2e800744e0 2024-04-25T19:03:05.893 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:05.894 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 msgr2=0x7f2e800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 0x7f2e800769f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009160 tx=0x7f2e94008040 comp rx=0 tx=0).stop 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 msgr2=0x7f2ea4157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ea415d1b0 tx=0x7f2e9c00e530 comp rx=0 tx=0).stop 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 shutdown_connections 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e800744e0 0x7f2e800769f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ea41fa510 0x7f2ea4157e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ea41f9bc0 0x7f2ea41f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 --2- 172.21.15.123:0/3956067577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88006a10 0x7f2ea4156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:05.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.892+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 >> 172.21.15.123:0/3956067577 conn(0x7f2ea4073200 msgr2=0x7f2ea40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:05.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.893+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 shutdown_connections 2024-04-25T19:03:05.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:05.893+0000 7f2ea3200640 1 -- 172.21.15.123:0/3956067577 wait complete. 2024-04-25T19:03:05.921 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T19:03:04.971011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219516,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755204,"statfs":{"total":773094113280,"available":772869328896,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006129"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137574+0000","last_change":"2024-04-25T19:02:39.052814+0000","last_active":"2024-04-25T19:02:45.137574+0000","last_peered":"2024-04-25T19:02:45.137574+0000","last_clean":"2024-04-25T19:02:45.137574+0000","last_became_active":"2024-04-25T19:02:39.052399+0000","last_became_peered":"2024-04-25T19:02:39.052399+0000","last_unstale":"2024-04-25T19:02:45.137574+0000","last_undegraded":"2024-04-25T19:02:45.137574+0000","last_fullsized":"2024-04-25T19:02:45.137574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202717+0000","last_change":"2024-04-25T19:02:40.072630+0000","last_active":"2024-04-25T19:02:45.202717+0000","last_peered":"2024-04-25T19:02:45.202717+0000","last_clean":"2024-04-25T19:02:45.202717+0000","last_became_active":"2024-04-25T19:02:40.072046+0000","last_became_peered":"2024-04-25T19:02:40.072046+0000","last_unstale":"2024-04-25T19:02:45.202717+0000","last_undegraded":"2024-04-25T19:02:45.202717+0000","last_fullsized":"2024-04-25T19:02:45.202717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200723+0000","last_change":"2024-04-25T19:02:44.161105+0000","last_active":"2024-04-25T19:02:45.200723+0000","last_peered":"2024-04-25T19:02:45.200723+0000","last_clean":"2024-04-25T19:02:45.200723+0000","last_became_active":"2024-04-25T19:02:44.160613+0000","last_became_peered":"2024-04-25T19:02:44.160613+0000","last_unstale":"2024-04-25T19:02:45.200723+0000","last_undegraded":"2024-04-25T19:02:45.200723+0000","last_fullsized":"2024-04-25T19:02:45.200723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:38.319660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203032+0000","last_change":"2024-04-25T19:02:39.042391+0000","last_active":"2024-04-25T19:02:45.203032+0000","last_peered":"2024-04-25T19:02:45.203032+0000","last_clean":"2024-04-25T19:02:45.203032+0000","last_became_active":"2024-04-25T19:02:39.042017+0000","last_became_peered":"2024-04-25T19:02:39.042017+0000","last_unstale":"2024-04-25T19:02:45.203032+0000","last_undegraded":"2024-04-25T19:02:45.203032+0000","last_fullsized":"2024-04-25T19:02:45.203032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203099+0000","last_change":"2024-04-25T19:02:40.073983+0000","last_active":"2024-04-25T19:02:45.203099+0000","last_peered":"2024-04-25T19:02:45.203099+0000","last_clean":"2024-04-25T19:02:45.203099+0000","last_became_active":"2024-04-25T19:02:40.069080+0000","last_became_peered":"2024-04-25T19:02:40.069080+0000","last_unstale":"2024-04-25T19:02:45.203099+0000","last_undegraded":"2024-04-25T19:02:45.203099+0000","last_fullsized":"2024-04-25T19:02:45.203099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198316+0000","last_change":"2024-04-25T19:02:44.148887+0000","last_active":"2024-04-25T19:02:45.198316+0000","last_peered":"2024-04-25T19:02:45.198316+0000","last_clean":"2024-04-25T19:02:45.198316+0000","last_became_active":"2024-04-25T19:02:44.148199+0000","last_became_peered":"2024-04-25T19:02:44.148199+0000","last_unstale":"2024-04-25T19:02:45.198316+0000","last_undegraded":"2024-04-25T19:02:45.198316+0000","last_fullsized":"2024-04-25T19:02:45.198316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:55.854080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199269+0000","last_change":"2024-04-25T19:02:39.047541+0000","last_active":"2024-04-25T19:02:45.199269+0000","last_peered":"2024-04-25T19:02:45.199269+0000","last_clean":"2024-04-25T19:02:45.199269+0000","last_became_active":"2024-04-25T19:02:39.047208+0000","last_became_peered":"2024-04-25T19:02:39.047208+0000","last_unstale":"2024-04-25T19:02:45.199269+0000","last_undegraded":"2024-04-25T19:02:45.199269+0000","last_fullsized":"2024-04-25T19:02:45.199269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200023+0000","last_change":"2024-04-25T19:02:40.076534+0000","last_active":"2024-04-25T19:02:45.200023+0000","last_peered":"2024-04-25T19:02:45.200023+0000","last_clean":"2024-04-25T19:02:45.200023+0000","last_became_active":"2024-04-25T19:02:40.076129+0000","last_became_peered":"2024-04-25T19:02:40.076129+0000","last_unstale":"2024-04-25T19:02:45.200023+0000","last_undegraded":"2024-04-25T19:02:45.200023+0000","last_fullsized":"2024-04-25T19:02:45.200023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198683+0000","last_change":"2024-04-25T19:02:44.153635+0000","last_active":"2024-04-25T19:02:45.198683+0000","last_peered":"2024-04-25T19:02:45.198683+0000","last_clean":"2024-04-25T19:02:45.198683+0000","last_became_active":"2024-04-25T19:02:44.153134+0000","last_became_peered":"2024-04-25T19:02:44.153134+0000","last_unstale":"2024-04-25T19:02:45.198683+0000","last_undegraded":"2024-04-25T19:02:45.198683+0000","last_fullsized":"2024-04-25T19:02:45.198683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:25.102590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199200+0000","last_change":"2024-04-25T19:02:39.047906+0000","last_active":"2024-04-25T19:02:45.199200+0000","last_peered":"2024-04-25T19:02:45.199200+0000","last_clean":"2024-04-25T19:02:45.199200+0000","last_became_active":"2024-04-25T19:02:39.044722+0000","last_became_peered":"2024-04-25T19:02:39.044722+0000","last_unstale":"2024-04-25T19:02:45.199200+0000","last_undegraded":"2024-04-25T19:02:45.199200+0000","last_fullsized":"2024-04-25T19:02:45.199200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130532+0000","last_change":"2024-04-25T19:02:40.084937+0000","last_active":"2024-04-25T19:02:45.130532+0000","last_peered":"2024-04-25T19:02:45.130532+0000","last_clean":"2024-04-25T19:02:45.130532+0000","last_became_active":"2024-04-25T19:02:40.084547+0000","last_became_peered":"2024-04-25T19:02:40.084547+0000","last_unstale":"2024-04-25T19:02:45.130532+0000","last_undegraded":"2024-04-25T19:02:45.130532+0000","last_fullsized":"2024-04-25T19:02:45.130532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:00.967270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197619+0000","last_change":"2024-04-25T19:02:44.152662+0000","last_active":"2024-04-25T19:02:45.197619+0000","last_peered":"2024-04-25T19:02:45.197619+0000","last_clean":"2024-04-25T19:02:45.197619+0000","last_became_active":"2024-04-25T19:02:44.152150+0000","last_became_peered":"2024-04-25T19:02:44.152150+0000","last_unstale":"2024-04-25T19:02:45.197619+0000","last_undegraded":"2024-04-25T19:02:45.197619+0000","last_fullsized":"2024-04-25T19:02:45.197619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:04.766506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129234+0000","last_change":"2024-04-25T19:02:40.069483+0000","last_active":"2024-04-25T19:02:45.129234+0000","last_peered":"2024-04-25T19:02:45.129234+0000","last_clean":"2024-04-25T19:02:45.129234+0000","last_became_active":"2024-04-25T19:02:40.068633+0000","last_became_peered":"2024-04-25T19:02:40.068633+0000","last_unstale":"2024-04-25T19:02:45.129234+0000","last_undegraded":"2024-04-25T19:02:45.129234+0000","last_fullsized":"2024-04-25T19:02:45.129234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:46.741238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130329+0000","last_change":"2024-04-25T19:02:39.036777+0000","last_active":"2024-04-25T19:02:45.130329+0000","last_peered":"2024-04-25T19:02:45.130329+0000","last_clean":"2024-04-25T19:02:45.130329+0000","last_became_active":"2024-04-25T19:02:39.036471+0000","last_became_peered":"2024-04-25T19:02:39.036471+0000","last_unstale":"2024-04-25T19:02:45.130329+0000","last_undegraded":"2024-04-25T19:02:45.130329+0000","last_fullsized":"2024-04-25T19:02:45.130329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:41.460879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130171+0000","last_change":"2024-04-25T19:02:44.153484+0000","last_active":"2024-04-25T19:02:45.130171+0000","last_peered":"2024-04-25T19:02:45.130171+0000","last_clean":"2024-04-25T19:02:45.130171+0000","last_became_active":"2024-04-25T19:02:44.153061+0000","last_became_peered":"2024-04-25T19:02:44.153061+0000","last_unstale":"2024-04-25T19:02:45.130171+0000","last_undegraded":"2024-04-25T19:02:45.130171+0000","last_fullsized":"2024-04-25T19:02:45.130171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:14.106695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201441+0000","last_change":"2024-04-25T19:02:40.071684+0000","last_active":"2024-04-25T19:02:45.201441+0000","last_peered":"2024-04-25T19:02:45.201441+0000","last_clean":"2024-04-25T19:02:45.201441+0000","last_became_active":"2024-04-25T19:02:40.071149+0000","last_became_peered":"2024-04-25T19:02:40.071149+0000","last_unstale":"2024-04-25T19:02:45.201441+0000","last_undegraded":"2024-04-25T19:02:45.201441+0000","last_fullsized":"2024-04-25T19:02:45.201441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134616+0000","last_change":"2024-04-25T19:02:39.047265+0000","last_active":"2024-04-25T19:02:45.134616+0000","last_peered":"2024-04-25T19:02:45.134616+0000","last_clean":"2024-04-25T19:02:45.134616+0000","last_became_active":"2024-04-25T19:02:39.046336+0000","last_became_peered":"2024-04-25T19:02:39.046336+0000","last_unstale":"2024-04-25T19:02:45.134616+0000","last_undegraded":"2024-04-25T19:02:45.134616+0000","last_fullsized":"2024-04-25T19:02:45.134616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:02.247449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198218+0000","last_change":"2024-04-25T19:02:44.139798+0000","last_active":"2024-04-25T19:02:45.198218+0000","last_peered":"2024-04-25T19:02:45.198218+0000","last_clean":"2024-04-25T19:02:45.198218+0000","last_became_active":"2024-04-25T19:02:44.139333+0000","last_became_peered":"2024-04-25T19:02:44.139333+0000","last_unstale":"2024-04-25T19:02:45.198218+0000","last_undegraded":"2024-04-25T19:02:45.198218+0000","last_fullsized":"2024-04-25T19:02:45.198218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:05.053484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131214+0000","last_change":"2024-04-25T19:02:40.065574+0000","last_active":"2024-04-25T19:02:45.131214+0000","last_peered":"2024-04-25T19:02:45.131214+0000","last_clean":"2024-04-25T19:02:45.131214+0000","last_became_active":"2024-04-25T19:02:40.060953+0000","last_became_peered":"2024-04-25T19:02:40.060953+0000","last_unstale":"2024-04-25T19:02:45.131214+0000","last_undegraded":"2024-04-25T19:02:45.131214+0000","last_fullsized":"2024-04-25T19:02:45.131214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132354+0000","last_change":"2024-04-25T19:02:39.044906+0000","last_active":"2024-04-25T19:02:45.132354+0000","last_peered":"2024-04-25T19:02:45.132354+0000","last_clean":"2024-04-25T19:02:45.132354+0000","last_became_active":"2024-04-25T19:02:39.043065+0000","last_became_peered":"2024-04-25T19:02:39.043065+0000","last_unstale":"2024-04-25T19:02:45.132354+0000","last_undegraded":"2024-04-25T19:02:45.132354+0000","last_fullsized":"2024-04-25T19:02:45.132354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203780+0000","last_change":"2024-04-25T19:02:44.149122+0000","last_active":"2024-04-25T19:02:45.203780+0000","last_peered":"2024-04-25T19:02:45.203780+0000","last_clean":"2024-04-25T19:02:45.203780+0000","last_became_active":"2024-04-25T19:02:44.148633+0000","last_became_peered":"2024-04-25T19:02:44.148633+0000","last_unstale":"2024-04-25T19:02:45.203780+0000","last_undegraded":"2024-04-25T19:02:45.203780+0000","last_fullsized":"2024-04-25T19:02:45.203780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128923+0000","last_change":"2024-04-25T19:02:40.067343+0000","last_active":"2024-04-25T19:02:45.128923+0000","last_peered":"2024-04-25T19:02:45.128923+0000","last_clean":"2024-04-25T19:02:45.128923+0000","last_became_active":"2024-04-25T19:02:40.066748+0000","last_became_peered":"2024-04-25T19:02:40.066748+0000","last_unstale":"2024-04-25T19:02:45.128923+0000","last_undegraded":"2024-04-25T19:02:45.128923+0000","last_fullsized":"2024-04-25T19:02:45.128923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:18.495508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196634+0000","last_change":"2024-04-25T19:02:39.042622+0000","last_active":"2024-04-25T19:02:45.196634+0000","last_peered":"2024-04-25T19:02:45.196634+0000","last_clean":"2024-04-25T19:02:45.196634+0000","last_became_active":"2024-04-25T19:02:39.042302+0000","last_became_peered":"2024-04-25T19:02:39.042302+0000","last_unstale":"2024-04-25T19:02:45.196634+0000","last_undegraded":"2024-04-25T19:02:45.196634+0000","last_fullsized":"2024-04-25T19:02:45.196634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199162+0000","last_change":"2024-04-25T19:02:44.148027+0000","last_active":"2024-04-25T19:02:45.199162+0000","last_peered":"2024-04-25T19:02:45.199162+0000","last_clean":"2024-04-25T19:02:45.199162+0000","last_became_active":"2024-04-25T19:02:44.147612+0000","last_became_peered":"2024-04-25T19:02:44.147612+0000","last_unstale":"2024-04-25T19:02:45.199162+0000","last_undegraded":"2024-04-25T19:02:45.199162+0000","last_fullsized":"2024-04-25T19:02:45.199162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:52.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136112+0000","last_change":"2024-04-25T19:02:40.062872+0000","last_active":"2024-04-25T19:02:45.136112+0000","last_peered":"2024-04-25T19:02:45.136112+0000","last_clean":"2024-04-25T19:02:45.136112+0000","last_became_active":"2024-04-25T19:02:40.061519+0000","last_became_peered":"2024-04-25T19:02:40.061519+0000","last_unstale":"2024-04-25T19:02:45.136112+0000","last_undegraded":"2024-04-25T19:02:45.136112+0000","last_fullsized":"2024-04-25T19:02:45.136112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132193+0000","last_change":"2024-04-25T19:02:39.054460+0000","last_active":"2024-04-25T19:02:45.132193+0000","last_peered":"2024-04-25T19:02:45.132193+0000","last_clean":"2024-04-25T19:02:45.132193+0000","last_became_active":"2024-04-25T19:02:39.053233+0000","last_became_peered":"2024-04-25T19:02:39.053233+0000","last_unstale":"2024-04-25T19:02:45.132193+0000","last_undegraded":"2024-04-25T19:02:45.132193+0000","last_fullsized":"2024-04-25T19:02:45.132193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:17.622604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199607+0000","last_change":"2024-04-25T19:02:44.139268+0000","last_active":"2024-04-25T19:02:45.199607+0000","last_peered":"2024-04-25T19:02:45.199607+0000","last_clean":"2024-04-25T19:02:45.199607+0000","last_became_active":"2024-04-25T19:02:44.137641+0000","last_became_peered":"2024-04-25T19:02:44.137641+0000","last_unstale":"2024-04-25T19:02:45.199607+0000","last_undegraded":"2024-04-25T19:02:45.199607+0000","last_fullsized":"2024-04-25T19:02:45.199607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137364+0000","last_change":"2024-04-25T19:02:40.063483+0000","last_active":"2024-04-25T19:02:45.137364+0000","last_peered":"2024-04-25T19:02:45.137364+0000","last_clean":"2024-04-25T19:02:45.137364+0000","last_became_active":"2024-04-25T19:02:40.062531+0000","last_became_peered":"2024-04-25T19:02:40.062531+0000","last_unstale":"2024-04-25T19:02:45.137364+0000","last_undegraded":"2024-04-25T19:02:45.137364+0000","last_fullsized":"2024-04-25T19:02:45.137364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136358+0000","last_change":"2024-04-25T19:02:39.052714+0000","last_active":"2024-04-25T19:02:45.136358+0000","last_peered":"2024-04-25T19:02:45.136358+0000","last_clean":"2024-04-25T19:02:45.136358+0000","last_became_active":"2024-04-25T19:02:39.051543+0000","last_became_peered":"2024-04-25T19:02:39.051543+0000","last_unstale":"2024-04-25T19:02:45.136358+0000","last_undegraded":"2024-04-25T19:02:45.136358+0000","last_fullsized":"2024-04-25T19:02:45.136358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131504+0000","last_change":"2024-04-25T19:02:44.142273+0000","last_active":"2024-04-25T19:02:45.131504+0000","last_peered":"2024-04-25T19:02:45.131504+0000","last_clean":"2024-04-25T19:02:45.131504+0000","last_became_active":"2024-04-25T19:02:44.140876+0000","last_became_peered":"2024-04-25T19:02:44.140876+0000","last_unstale":"2024-04-25T19:02:45.131504+0000","last_undegraded":"2024-04-25T19:02:45.131504+0000","last_fullsized":"2024-04-25T19:02:45.131504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137451+0000","last_change":"2024-04-25T19:02:40.065602+0000","last_active":"2024-04-25T19:02:45.137451+0000","last_peered":"2024-04-25T19:02:45.137451+0000","last_clean":"2024-04-25T19:02:45.137451+0000","last_became_active":"2024-04-25T19:02:40.065178+0000","last_became_peered":"2024-04-25T19:02:40.065178+0000","last_unstale":"2024-04-25T19:02:45.137451+0000","last_undegraded":"2024-04-25T19:02:45.137451+0000","last_fullsized":"2024-04-25T19:02:45.137451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.101532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131078+0000","last_change":"2024-04-25T19:02:39.054975+0000","last_active":"2024-04-25T19:02:45.131078+0000","last_peered":"2024-04-25T19:02:45.131078+0000","last_clean":"2024-04-25T19:02:45.131078+0000","last_became_active":"2024-04-25T19:02:39.052591+0000","last_became_peered":"2024-04-25T19:02:39.052591+0000","last_unstale":"2024-04-25T19:02:45.131078+0000","last_undegraded":"2024-04-25T19:02:45.131078+0000","last_fullsized":"2024-04-25T19:02:45.131078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:10.725639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135617+0000","last_change":"2024-04-25T19:02:44.144565+0000","last_active":"2024-04-25T19:02:45.135617+0000","last_peered":"2024-04-25T19:02:45.135617+0000","last_clean":"2024-04-25T19:02:45.135617+0000","last_became_active":"2024-04-25T19:02:44.144092+0000","last_became_peered":"2024-04-25T19:02:44.144092+0000","last_unstale":"2024-04-25T19:02:45.135617+0000","last_undegraded":"2024-04-25T19:02:45.135617+0000","last_fullsized":"2024-04-25T19:02:45.135617+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:18.177897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198835+0000","last_change":"2024-04-25T19:02:40.062748+0000","last_active":"2024-04-25T19:02:45.198835+0000","last_peered":"2024-04-25T19:02:45.198835+0000","last_clean":"2024-04-25T19:02:45.198835+0000","last_became_active":"2024-04-25T19:02:40.060618+0000","last_became_peered":"2024-04-25T19:02:40.060618+0000","last_unstale":"2024-04-25T19:02:45.198835+0000","last_undegraded":"2024-04-25T19:02:45.198835+0000","last_fullsized":"2024-04-25T19:02:45.198835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200394+0000","last_change":"2024-04-25T19:02:39.043998+0000","last_active":"2024-04-25T19:02:45.200394+0000","last_peered":"2024-04-25T19:02:45.200394+0000","last_clean":"2024-04-25T19:02:45.200394+0000","last_became_active":"2024-04-25T19:02:39.043623+0000","last_became_peered":"2024-04-25T19:02:39.043623+0000","last_unstale":"2024-04-25T19:02:45.200394+0000","last_undegraded":"2024-04-25T19:02:45.200394+0000","last_fullsized":"2024-04-25T19:02:45.200394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133644+0000","last_change":"2024-04-25T19:02:44.158919+0000","last_active":"2024-04-25T19:02:45.133644+0000","last_peered":"2024-04-25T19:02:45.133644+0000","last_clean":"2024-04-25T19:02:45.133644+0000","last_became_active":"2024-04-25T19:02:44.158358+0000","last_became_peered":"2024-04-25T19:02:44.158358+0000","last_unstale":"2024-04-25T19:02:45.133644+0000","last_undegraded":"2024-04-25T19:02:45.133644+0000","last_fullsized":"2024-04-25T19:02:45.133644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:05.831945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198482+0000","last_change":"2024-04-25T19:02:40.072547+0000","last_active":"2024-04-25T19:02:45.198482+0000","last_peered":"2024-04-25T19:02:45.198482+0000","last_clean":"2024-04-25T19:02:45.198482+0000","last_became_active":"2024-04-25T19:02:40.072124+0000","last_became_peered":"2024-04-25T19:02:40.072124+0000","last_unstale":"2024-04-25T19:02:45.198482+0000","last_undegraded":"2024-04-25T19:02:45.198482+0000","last_fullsized":"2024-04-25T19:02:45.198482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130699+0000","last_change":"2024-04-25T19:02:39.051040+0000","last_active":"2024-04-25T19:02:45.130699+0000","last_peered":"2024-04-25T19:02:45.130699+0000","last_clean":"2024-04-25T19:02:45.130699+0000","last_became_active":"2024-04-25T19:02:39.048647+0000","last_became_peered":"2024-04-25T19:02:39.048647+0000","last_unstale":"2024-04-25T19:02:45.130699+0000","last_undegraded":"2024-04-25T19:02:45.130699+0000","last_fullsized":"2024-04-25T19:02:45.130699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:53.187342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136962+0000","last_change":"2024-04-25T19:02:44.151392+0000","last_active":"2024-04-25T19:02:45.136962+0000","last_peered":"2024-04-25T19:02:45.136962+0000","last_clean":"2024-04-25T19:02:45.136962+0000","last_became_active":"2024-04-25T19:02:44.150074+0000","last_became_peered":"2024-04-25T19:02:44.150074+0000","last_unstale":"2024-04-25T19:02:45.136962+0000","last_undegraded":"2024-04-25T19:02:45.136962+0000","last_fullsized":"2024-04-25T19:02:45.136962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:03.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199109+0000","last_change":"2024-04-25T19:02:40.074602+0000","last_active":"2024-04-25T19:02:45.199109+0000","last_peered":"2024-04-25T19:02:45.199109+0000","last_clean":"2024-04-25T19:02:45.199109+0000","last_became_active":"2024-04-25T19:02:40.070530+0000","last_became_peered":"2024-04-25T19:02:40.070530+0000","last_unstale":"2024-04-25T19:02:45.199109+0000","last_undegraded":"2024-04-25T19:02:45.199109+0000","last_fullsized":"2024-04-25T19:02:45.199109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200966+0000","last_change":"2024-04-25T19:02:39.048859+0000","last_active":"2024-04-25T19:02:45.200966+0000","last_peered":"2024-04-25T19:02:45.200966+0000","last_clean":"2024-04-25T19:02:45.200966+0000","last_became_active":"2024-04-25T19:02:39.048497+0000","last_became_peered":"2024-04-25T19:02:39.048497+0000","last_unstale":"2024-04-25T19:02:45.200966+0000","last_undegraded":"2024-04-25T19:02:45.200966+0000","last_fullsized":"2024-04-25T19:02:45.200966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:13.519715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137567+0000","last_change":"2024-04-25T19:02:44.150754+0000","last_active":"2024-04-25T19:02:45.137567+0000","last_peered":"2024-04-25T19:02:45.137567+0000","last_clean":"2024-04-25T19:02:45.137567+0000","last_became_active":"2024-04-25T19:02:44.150092+0000","last_became_peered":"2024-04-25T19:02:44.150092+0000","last_unstale":"2024-04-25T19:02:45.137567+0000","last_undegraded":"2024-04-25T19:02:45.137567+0000","last_fullsized":"2024-04-25T19:02:45.137567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:45.602374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132688+0000","last_change":"2024-04-25T19:02:40.068029+0000","last_active":"2024-04-25T19:02:45.132688+0000","last_peered":"2024-04-25T19:02:45.132688+0000","last_clean":"2024-04-25T19:02:45.132688+0000","last_became_active":"2024-04-25T19:02:40.066355+0000","last_became_peered":"2024-04-25T19:02:40.066355+0000","last_unstale":"2024-04-25T19:02:45.132688+0000","last_undegraded":"2024-04-25T19:02:45.132688+0000","last_fullsized":"2024-04-25T19:02:45.132688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:09.936302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129192+0000","last_change":"2024-04-25T19:02:39.047056+0000","last_active":"2024-04-25T19:02:45.129192+0000","last_peered":"2024-04-25T19:02:45.129192+0000","last_clean":"2024-04-25T19:02:45.129192+0000","last_became_active":"2024-04-25T19:02:39.046198+0000","last_became_peered":"2024-04-25T19:02:39.046198+0000","last_unstale":"2024-04-25T19:02:45.129192+0000","last_undegraded":"2024-04-25T19:02:45.129192+0000","last_fullsized":"2024-04-25T19:02:45.129192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202550+0000","last_change":"2024-04-25T19:02:44.158508+0000","last_active":"2024-04-25T19:02:45.202550+0000","last_peered":"2024-04-25T19:02:45.202550+0000","last_clean":"2024-04-25T19:02:45.202550+0000","last_became_active":"2024-04-25T19:02:44.157734+0000","last_became_peered":"2024-04-25T19:02:44.157734+0000","last_unstale":"2024-04-25T19:02:45.202550+0000","last_undegraded":"2024-04-25T19:02:45.202550+0000","last_fullsized":"2024-04-25T19:02:45.202550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:42.894635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133801+0000","last_change":"2024-04-25T19:02:40.048821+0000","last_active":"2024-04-25T19:02:45.133801+0000","last_peered":"2024-04-25T19:02:45.133801+0000","last_clean":"2024-04-25T19:02:45.133801+0000","last_became_active":"2024-04-25T19:02:40.048295+0000","last_became_peered":"2024-04-25T19:02:40.048295+0000","last_unstale":"2024-04-25T19:02:45.133801+0000","last_undegraded":"2024-04-25T19:02:45.133801+0000","last_fullsized":"2024-04-25T19:02:45.133801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129911+0000","last_change":"2024-04-25T19:02:39.036175+0000","last_active":"2024-04-25T19:02:45.129911+0000","last_peered":"2024-04-25T19:02:45.129911+0000","last_clean":"2024-04-25T19:02:45.129911+0000","last_became_active":"2024-04-25T19:02:39.035822+0000","last_became_peered":"2024-04-25T19:02:39.035822+0000","last_unstale":"2024-04-25T19:02:45.129911+0000","last_undegraded":"2024-04-25T19:02:45.129911+0000","last_fullsized":"2024-04-25T19:02:45.129911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.306938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197039+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:02:45.197039+0000","last_peered":"2024-04-25T19:02:45.197039+0000","last_clean":"2024-04-25T19:02:45.197039+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:02:45.197039+0000","last_undegraded":"2024-04-25T19:02:45.197039+0000","last_fullsized":"2024-04-25T19:02:45.197039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131234+0000","last_change":"2024-04-25T19:02:44.144534+0000","last_active":"2024-04-25T19:02:45.131234+0000","last_peered":"2024-04-25T19:02:45.131234+0000","last_clean":"2024-04-25T19:02:45.131234+0000","last_became_active":"2024-04-25T19:02:44.143365+0000","last_became_peered":"2024-04-25T19:02:44.143365+0000","last_unstale":"2024-04-25T19:02:45.131234+0000","last_undegraded":"2024-04-25T19:02:45.131234+0000","last_fullsized":"2024-04-25T19:02:45.131234+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:39.667305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130665+0000","last_change":"2024-04-25T19:02:40.082915+0000","last_active":"2024-04-25T19:02:45.130665+0000","last_peered":"2024-04-25T19:02:45.130665+0000","last_clean":"2024-04-25T19:02:45.130665+0000","last_became_active":"2024-04-25T19:02:40.082594+0000","last_became_peered":"2024-04-25T19:02:40.082594+0000","last_unstale":"2024-04-25T19:02:45.130665+0000","last_undegraded":"2024-04-25T19:02:45.130665+0000","last_fullsized":"2024-04-25T19:02:45.130665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:10.588191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137737+0000","last_change":"2024-04-25T19:02:39.052990+0000","last_active":"2024-04-25T19:02:45.137737+0000","last_peered":"2024-04-25T19:02:45.137737+0000","last_clean":"2024-04-25T19:02:45.137737+0000","last_became_active":"2024-04-25T19:02:39.052635+0000","last_became_peered":"2024-04-25T19:02:39.052635+0000","last_unstale":"2024-04-25T19:02:45.137737+0000","last_undegraded":"2024-04-25T19:02:45.137737+0000","last_fullsized":"2024-04-25T19:02:45.137737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.298378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198050+0000","last_change":"2024-04-25T19:02:44.143773+0000","last_active":"2024-04-25T19:02:45.198050+0000","last_peered":"2024-04-25T19:02:45.198050+0000","last_clean":"2024-04-25T19:02:45.198050+0000","last_became_active":"2024-04-25T19:02:44.141602+0000","last_became_peered":"2024-04-25T19:02:44.141602+0000","last_unstale":"2024-04-25T19:02:45.198050+0000","last_undegraded":"2024-04-25T19:02:45.198050+0000","last_fullsized":"2024-04-25T19:02:45.198050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:03.052112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199104+0000","last_change":"2024-04-25T19:02:40.075177+0000","last_active":"2024-04-25T19:02:45.199104+0000","last_peered":"2024-04-25T19:02:45.199104+0000","last_clean":"2024-04-25T19:02:45.199104+0000","last_became_active":"2024-04-25T19:02:40.074512+0000","last_became_peered":"2024-04-25T19:02:40.074512+0000","last_unstale":"2024-04-25T19:02:45.199104+0000","last_undegraded":"2024-04-25T19:02:45.199104+0000","last_fullsized":"2024-04-25T19:02:45.199104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201661+0000","last_change":"2024-04-25T19:02:39.052054+0000","last_active":"2024-04-25T19:02:45.201661+0000","last_peered":"2024-04-25T19:02:45.201661+0000","last_clean":"2024-04-25T19:02:45.201661+0000","last_became_active":"2024-04-25T19:02:39.051646+0000","last_became_peered":"2024-04-25T19:02:39.051646+0000","last_unstale":"2024-04-25T19:02:45.201661+0000","last_undegraded":"2024-04-25T19:02:45.201661+0000","last_fullsized":"2024-04-25T19:02:45.201661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200618+0000","last_change":"2024-04-25T19:02:44.157149+0000","last_active":"2024-04-25T19:02:45.200618+0000","last_peered":"2024-04-25T19:02:45.200618+0000","last_clean":"2024-04-25T19:02:45.200618+0000","last_became_active":"2024-04-25T19:02:44.156656+0000","last_became_peered":"2024-04-25T19:02:44.156656+0000","last_unstale":"2024-04-25T19:02:45.200618+0000","last_undegraded":"2024-04-25T19:02:45.200618+0000","last_fullsized":"2024-04-25T19:02:45.200618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:14.674405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198854+0000","last_change":"2024-04-25T19:02:40.080039+0000","last_active":"2024-04-25T19:02:45.198854+0000","last_peered":"2024-04-25T19:02:45.198854+0000","last_clean":"2024-04-25T19:02:45.198854+0000","last_became_active":"2024-04-25T19:02:40.079725+0000","last_became_peered":"2024-04-25T19:02:40.079725+0000","last_unstale":"2024-04-25T19:02:45.198854+0000","last_undegraded":"2024-04-25T19:02:45.198854+0000","last_fullsized":"2024-04-25T19:02:45.198854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136108+0000","last_change":"2024-04-25T19:02:39.051841+0000","last_active":"2024-04-25T19:02:45.136108+0000","last_peered":"2024-04-25T19:02:45.136108+0000","last_clean":"2024-04-25T19:02:45.136108+0000","last_became_active":"2024-04-25T19:02:39.050695+0000","last_became_peered":"2024-04-25T19:02:39.050695+0000","last_unstale":"2024-04-25T19:02:45.136108+0000","last_undegraded":"2024-04-25T19:02:45.136108+0000","last_fullsized":"2024-04-25T19:02:45.136108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128669+0000","last_change":"2024-04-25T19:02:44.147067+0000","last_active":"2024-04-25T19:02:45.128669+0000","last_peered":"2024-04-25T19:02:45.128669+0000","last_clean":"2024-04-25T19:02:45.128669+0000","last_became_active":"2024-04-25T19:02:44.146048+0000","last_became_peered":"2024-04-25T19:02:44.146048+0000","last_unstale":"2024-04-25T19:02:45.128669+0000","last_undegraded":"2024-04-25T19:02:45.128669+0000","last_fullsized":"2024-04-25T19:02:45.128669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:13.605566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199716+0000","last_change":"2024-04-25T19:02:40.066529+0000","last_active":"2024-04-25T19:02:45.199716+0000","last_peered":"2024-04-25T19:02:45.199716+0000","last_clean":"2024-04-25T19:02:45.199716+0000","last_became_active":"2024-04-25T19:02:40.064814+0000","last_became_peered":"2024-04-25T19:02:40.064814+0000","last_unstale":"2024-04-25T19:02:45.199716+0000","last_undegraded":"2024-04-25T19:02:45.199716+0000","last_fullsized":"2024-04-25T19:02:45.199716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199020+0000","last_change":"2024-04-25T19:02:39.051220+0000","last_active":"2024-04-25T19:02:45.199020+0000","last_peered":"2024-04-25T19:02:45.199020+0000","last_clean":"2024-04-25T19:02:45.199020+0000","last_became_active":"2024-04-25T19:02:39.050911+0000","last_became_peered":"2024-04-25T19:02:39.050911+0000","last_unstale":"2024-04-25T19:02:45.199020+0000","last_undegraded":"2024-04-25T19:02:45.199020+0000","last_fullsized":"2024-04-25T19:02:45.199020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196897+0000","last_change":"2024-04-25T19:02:44.156822+0000","last_active":"2024-04-25T19:02:45.196897+0000","last_peered":"2024-04-25T19:02:45.196897+0000","last_clean":"2024-04-25T19:02:45.196897+0000","last_became_active":"2024-04-25T19:02:44.156379+0000","last_became_peered":"2024-04-25T19:02:44.156379+0000","last_unstale":"2024-04-25T19:02:45.196897+0000","last_undegraded":"2024-04-25T19:02:45.196897+0000","last_fullsized":"2024-04-25T19:02:45.196897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128832+0000","last_change":"2024-04-25T19:02:40.075607+0000","last_active":"2024-04-25T19:02:45.128832+0000","last_peered":"2024-04-25T19:02:45.128832+0000","last_clean":"2024-04-25T19:02:45.128832+0000","last_became_active":"2024-04-25T19:02:40.075267+0000","last_became_peered":"2024-04-25T19:02:40.075267+0000","last_unstale":"2024-04-25T19:02:45.128832+0000","last_undegraded":"2024-04-25T19:02:45.128832+0000","last_fullsized":"2024-04-25T19:02:45.128832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131535+0000","last_change":"2024-04-25T19:02:39.044744+0000","last_active":"2024-04-25T19:02:45.131535+0000","last_peered":"2024-04-25T19:02:45.131535+0000","last_clean":"2024-04-25T19:02:45.131535+0000","last_became_active":"2024-04-25T19:02:39.044372+0000","last_became_peered":"2024-04-25T19:02:39.044372+0000","last_unstale":"2024-04-25T19:02:45.131535+0000","last_undegraded":"2024-04-25T19:02:45.131535+0000","last_fullsized":"2024-04-25T19:02:45.131535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203015+0000","last_change":"2024-04-25T19:02:44.158146+0000","last_active":"2024-04-25T19:02:45.203015+0000","last_peered":"2024-04-25T19:02:45.203015+0000","last_clean":"2024-04-25T19:02:45.203015+0000","last_became_active":"2024-04-25T19:02:44.157017+0000","last_became_peered":"2024-04-25T19:02:44.157017+0000","last_unstale":"2024-04-25T19:02:45.203015+0000","last_undegraded":"2024-04-25T19:02:45.203015+0000","last_fullsized":"2024-04-25T19:02:45.203015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:40.012791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196618+0000","last_change":"2024-04-25T19:02:40.073754+0000","last_active":"2024-04-25T19:02:45.196618+0000","last_peered":"2024-04-25T19:02:45.196618+0000","last_clean":"2024-04-25T19:02:45.196618+0000","last_became_active":"2024-04-25T19:02:40.073500+0000","last_became_peered":"2024-04-25T19:02:40.073500+0000","last_unstale":"2024-04-25T19:02:45.196618+0000","last_undegraded":"2024-04-25T19:02:45.196618+0000","last_fullsized":"2024-04-25T19:02:45.196618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197302+0000","last_change":"2024-04-25T19:02:39.039435+0000","last_active":"2024-04-25T19:02:45.197302+0000","last_peered":"2024-04-25T19:02:45.197302+0000","last_clean":"2024-04-25T19:02:45.197302+0000","last_became_active":"2024-04-25T19:02:39.039084+0000","last_became_peered":"2024-04-25T19:02:39.039084+0000","last_unstale":"2024-04-25T19:02:45.197302+0000","last_undegraded":"2024-04-25T19:02:45.197302+0000","last_fullsized":"2024-04-25T19:02:45.197302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202918+0000","last_change":"2024-04-25T19:02:44.156161+0000","last_active":"2024-04-25T19:02:45.202918+0000","last_peered":"2024-04-25T19:02:45.202918+0000","last_clean":"2024-04-25T19:02:45.202918+0000","last_became_active":"2024-04-25T19:02:44.155666+0000","last_became_peered":"2024-04-25T19:02:44.155666+0000","last_unstale":"2024-04-25T19:02:45.202918+0000","last_undegraded":"2024-04-25T19:02:45.202918+0000","last_fullsized":"2024-04-25T19:02:45.202918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:22:00.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129573+0000","last_change":"2024-04-25T19:02:40.076702+0000","last_active":"2024-04-25T19:02:45.129573+0000","last_peered":"2024-04-25T19:02:45.129573+0000","last_clean":"2024-04-25T19:02:45.129573+0000","last_became_active":"2024-04-25T19:02:40.072663+0000","last_became_peered":"2024-04-25T19:02:40.072663+0000","last_unstale":"2024-04-25T19:02:45.129573+0000","last_undegraded":"2024-04-25T19:02:45.129573+0000","last_fullsized":"2024-04-25T19:02:45.129573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:35.926577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130496+0000","last_change":"2024-04-25T19:02:39.048307+0000","last_active":"2024-04-25T19:02:45.130496+0000","last_peered":"2024-04-25T19:02:45.130496+0000","last_clean":"2024-04-25T19:02:45.130496+0000","last_became_active":"2024-04-25T19:02:39.047902+0000","last_became_peered":"2024-04-25T19:02:39.047902+0000","last_unstale":"2024-04-25T19:02:45.130496+0000","last_undegraded":"2024-04-25T19:02:45.130496+0000","last_fullsized":"2024-04-25T19:02:45.130496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136907+0000","last_change":"2024-04-25T19:02:44.151849+0000","last_active":"2024-04-25T19:02:45.136907+0000","last_peered":"2024-04-25T19:02:45.136907+0000","last_clean":"2024-04-25T19:02:45.136907+0000","last_became_active":"2024-04-25T19:02:44.150781+0000","last_became_peered":"2024-04-25T19:02:44.150781+0000","last_unstale":"2024-04-25T19:02:45.136907+0000","last_undegraded":"2024-04-25T19:02:45.136907+0000","last_fullsized":"2024-04-25T19:02:45.136907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:30.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134288+0000","last_change":"2024-04-25T19:02:40.073292+0000","last_active":"2024-04-25T19:02:45.134288+0000","last_peered":"2024-04-25T19:02:45.134288+0000","last_clean":"2024-04-25T19:02:45.134288+0000","last_became_active":"2024-04-25T19:02:40.070890+0000","last_became_peered":"2024-04-25T19:02:40.070890+0000","last_unstale":"2024-04-25T19:02:45.134288+0000","last_undegraded":"2024-04-25T19:02:45.134288+0000","last_fullsized":"2024-04-25T19:02:45.134288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:19.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201740+0000","last_change":"2024-04-25T19:02:39.053287+0000","last_active":"2024-04-25T19:02:45.201740+0000","last_peered":"2024-04-25T19:02:45.201740+0000","last_clean":"2024-04-25T19:02:45.201740+0000","last_became_active":"2024-04-25T19:02:39.052037+0000","last_became_peered":"2024-04-25T19:02:39.052037+0000","last_unstale":"2024-04-25T19:02:45.201740+0000","last_undegraded":"2024-04-25T19:02:45.201740+0000","last_fullsized":"2024-04-25T19:02:45.201740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130560+0000","last_change":"2024-04-25T19:02:44.158840+0000","last_active":"2024-04-25T19:02:45.130560+0000","last_peered":"2024-04-25T19:02:45.130560+0000","last_clean":"2024-04-25T19:02:45.130560+0000","last_became_active":"2024-04-25T19:02:44.156508+0000","last_became_peered":"2024-04-25T19:02:44.156508+0000","last_unstale":"2024-04-25T19:02:45.130560+0000","last_undegraded":"2024-04-25T19:02:45.130560+0000","last_fullsized":"2024-04-25T19:02:45.130560+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:08.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198990+0000","last_change":"2024-04-25T19:02:40.084277+0000","last_active":"2024-04-25T19:02:45.198990+0000","last_peered":"2024-04-25T19:02:45.198990+0000","last_clean":"2024-04-25T19:02:45.198990+0000","last_became_active":"2024-04-25T19:02:40.083896+0000","last_became_peered":"2024-04-25T19:02:40.083896+0000","last_unstale":"2024-04-25T19:02:45.198990+0000","last_undegraded":"2024-04-25T19:02:45.198990+0000","last_fullsized":"2024-04-25T19:02:45.198990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.393874+0000","last_change":"2024-04-25T19:02:39.045230+0000","last_active":"2024-04-25T19:03:00.393874+0000","last_peered":"2024-04-25T19:03:00.393874+0000","last_clean":"2024-04-25T19:03:00.393874+0000","last_became_active":"2024-04-25T19:02:39.044884+0000","last_became_peered":"2024-04-25T19:02:39.044884+0000","last_unstale":"2024-04-25T19:03:00.393874+0000","last_undegraded":"2024-04-25T19:03:00.393874+0000","last_fullsized":"2024-04-25T19:03:00.393874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:20.905809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133971+0000","last_change":"2024-04-25T19:02:44.153433+0000","last_active":"2024-04-25T19:02:45.133971+0000","last_peered":"2024-04-25T19:02:45.133971+0000","last_clean":"2024-04-25T19:02:45.133971+0000","last_became_active":"2024-04-25T19:02:44.152935+0000","last_became_peered":"2024-04-25T19:02:44.152935+0000","last_unstale":"2024-04-25T19:02:45.133971+0000","last_undegraded":"2024-04-25T19:02:45.133971+0000","last_fullsized":"2024-04-25T19:02:45.133971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:35.591324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129614+0000","last_change":"2024-04-25T19:02:40.074340+0000","last_active":"2024-04-25T19:02:45.129614+0000","last_peered":"2024-04-25T19:02:45.129614+0000","last_clean":"2024-04-25T19:02:45.129614+0000","last_became_active":"2024-04-25T19:02:40.069150+0000","last_became_peered":"2024-04-25T19:02:40.069150+0000","last_unstale":"2024-04-25T19:02:45.129614+0000","last_undegraded":"2024-04-25T19:02:45.129614+0000","last_fullsized":"2024-04-25T19:02:45.129614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:53.328375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203162+0000","last_change":"2024-04-25T19:02:39.050352+0000","last_active":"2024-04-25T19:02:45.203162+0000","last_peered":"2024-04-25T19:02:45.203162+0000","last_clean":"2024-04-25T19:02:45.203162+0000","last_became_active":"2024-04-25T19:02:39.049837+0000","last_became_peered":"2024-04-25T19:02:39.049837+0000","last_unstale":"2024-04-25T19:02:45.203162+0000","last_undegraded":"2024-04-25T19:02:45.203162+0000","last_fullsized":"2024-04-25T19:02:45.203162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:46.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200045+0000","last_change":"2024-04-25T19:02:44.147062+0000","last_active":"2024-04-25T19:02:45.200045+0000","last_peered":"2024-04-25T19:02:45.200045+0000","last_clean":"2024-04-25T19:02:45.200045+0000","last_became_active":"2024-04-25T19:02:44.145997+0000","last_became_peered":"2024-04-25T19:02:44.145997+0000","last_unstale":"2024-04-25T19:02:45.200045+0000","last_undegraded":"2024-04-25T19:02:45.200045+0000","last_fullsized":"2024-04-25T19:02:45.200045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203397+0000","last_change":"2024-04-25T19:02:40.083756+0000","last_active":"2024-04-25T19:02:45.203397+0000","last_peered":"2024-04-25T19:02:45.203397+0000","last_clean":"2024-04-25T19:02:45.203397+0000","last_became_active":"2024-04-25T19:02:40.083402+0000","last_became_peered":"2024-04-25T19:02:40.083402+0000","last_unstale":"2024-04-25T19:02:45.203397+0000","last_undegraded":"2024-04-25T19:02:45.203397+0000","last_fullsized":"2024-04-25T19:02:45.203397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198142+0000","last_change":"2024-04-25T19:02:39.043121+0000","last_active":"2024-04-25T19:02:45.198142+0000","last_peered":"2024-04-25T19:02:45.198142+0000","last_clean":"2024-04-25T19:02:45.198142+0000","last_became_active":"2024-04-25T19:02:39.042778+0000","last_became_peered":"2024-04-25T19:02:39.042778+0000","last_unstale":"2024-04-25T19:02:45.198142+0000","last_undegraded":"2024-04-25T19:02:45.198142+0000","last_fullsized":"2024-04-25T19:02:45.198142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199456+0000","last_change":"2024-04-25T19:02:44.148628+0000","last_active":"2024-04-25T19:02:45.199456+0000","last_peered":"2024-04-25T19:02:45.199456+0000","last_clean":"2024-04-25T19:02:45.199456+0000","last_became_active":"2024-04-25T19:02:44.147929+0000","last_became_peered":"2024-04-25T19:02:44.147929+0000","last_unstale":"2024-04-25T19:02:45.199456+0000","last_undegraded":"2024-04-25T19:02:45.199456+0000","last_fullsized":"2024-04-25T19:02:45.199456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199371+0000","last_change":"2024-04-25T19:02:40.069980+0000","last_active":"2024-04-25T19:02:45.199371+0000","last_peered":"2024-04-25T19:02:45.199371+0000","last_clean":"2024-04-25T19:02:45.199371+0000","last_became_active":"2024-04-25T19:02:40.069323+0000","last_became_peered":"2024-04-25T19:02:40.069323+0000","last_unstale":"2024-04-25T19:02:45.199371+0000","last_undegraded":"2024-04-25T19:02:45.199371+0000","last_fullsized":"2024-04-25T19:02:45.199371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200662+0000","last_change":"2024-04-25T19:02:39.044027+0000","last_active":"2024-04-25T19:02:45.200662+0000","last_peered":"2024-04-25T19:02:45.200662+0000","last_clean":"2024-04-25T19:02:45.200662+0000","last_became_active":"2024-04-25T19:02:39.043696+0000","last_became_peered":"2024-04-25T19:02:39.043696+0000","last_unstale":"2024-04-25T19:02:45.200662+0000","last_undegraded":"2024-04-25T19:02:45.200662+0000","last_fullsized":"2024-04-25T19:02:45.200662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132192+0000","last_change":"2024-04-25T19:02:44.147513+0000","last_active":"2024-04-25T19:02:45.132192+0000","last_peered":"2024-04-25T19:02:45.132192+0000","last_clean":"2024-04-25T19:02:45.132192+0000","last_became_active":"2024-04-25T19:02:44.146937+0000","last_became_peered":"2024-04-25T19:02:44.146937+0000","last_unstale":"2024-04-25T19:02:45.132192+0000","last_undegraded":"2024-04-25T19:02:45.132192+0000","last_fullsized":"2024-04-25T19:02:45.132192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135996+0000","last_change":"2024-04-25T19:02:44.153340+0000","last_active":"2024-04-25T19:02:45.135996+0000","last_peered":"2024-04-25T19:02:45.135996+0000","last_clean":"2024-04-25T19:02:45.135996+0000","last_became_active":"2024-04-25T19:02:44.152850+0000","last_became_peered":"2024-04-25T19:02:44.152850+0000","last_unstale":"2024-04-25T19:02:45.135996+0000","last_undegraded":"2024-04-25T19:02:45.135996+0000","last_fullsized":"2024-04-25T19:02:45.135996+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:57.414588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132085+0000","last_change":"2024-04-25T19:02:40.051844+0000","last_active":"2024-04-25T19:02:45.132085+0000","last_peered":"2024-04-25T19:02:45.132085+0000","last_clean":"2024-04-25T19:02:45.132085+0000","last_became_active":"2024-04-25T19:02:40.050922+0000","last_became_peered":"2024-04-25T19:02:40.050922+0000","last_unstale":"2024-04-25T19:02:45.132085+0000","last_undegraded":"2024-04-25T19:02:45.132085+0000","last_fullsized":"2024-04-25T19:02:45.132085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.344370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129179+0000","last_change":"2024-04-25T19:02:39.045270+0000","last_active":"2024-04-25T19:02:45.129179+0000","last_peered":"2024-04-25T19:02:45.129179+0000","last_clean":"2024-04-25T19:02:45.129179+0000","last_became_active":"2024-04-25T19:02:39.044936+0000","last_became_peered":"2024-04-25T19:02:39.044936+0000","last_unstale":"2024-04-25T19:02:45.129179+0000","last_undegraded":"2024-04-25T19:02:45.129179+0000","last_fullsized":"2024-04-25T19:02:45.129179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131178+0000","last_change":"2024-04-25T19:02:44.160881+0000","last_active":"2024-04-25T19:02:45.131178+0000","last_peered":"2024-04-25T19:02:45.131178+0000","last_clean":"2024-04-25T19:02:45.131178+0000","last_became_active":"2024-04-25T19:02:44.160267+0000","last_became_peered":"2024-04-25T19:02:44.160267+0000","last_unstale":"2024-04-25T19:02:45.131178+0000","last_undegraded":"2024-04-25T19:02:45.131178+0000","last_fullsized":"2024-04-25T19:02:45.131178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:21.942974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199361+0000","last_change":"2024-04-25T19:02:40.065898+0000","last_active":"2024-04-25T19:02:45.199361+0000","last_peered":"2024-04-25T19:02:45.199361+0000","last_clean":"2024-04-25T19:02:45.199361+0000","last_became_active":"2024-04-25T19:02:40.062397+0000","last_became_peered":"2024-04-25T19:02:40.062397+0000","last_unstale":"2024-04-25T19:02:45.199361+0000","last_undegraded":"2024-04-25T19:02:45.199361+0000","last_fullsized":"2024-04-25T19:02:45.199361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132878+0000","last_change":"2024-04-25T19:02:39.049593+0000","last_active":"2024-04-25T19:02:45.132878+0000","last_peered":"2024-04-25T19:02:45.132878+0000","last_clean":"2024-04-25T19:02:45.132878+0000","last_became_active":"2024-04-25T19:02:39.049140+0000","last_became_peered":"2024-04-25T19:02:39.049140+0000","last_unstale":"2024-04-25T19:02:45.132878+0000","last_undegraded":"2024-04-25T19:02:45.132878+0000","last_fullsized":"2024-04-25T19:02:45.132878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:01.624014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130188+0000","last_change":"2024-04-25T19:02:39.050611+0000","last_active":"2024-04-25T19:02:45.130188+0000","last_peered":"2024-04-25T19:02:45.130188+0000","last_clean":"2024-04-25T19:02:45.130188+0000","last_became_active":"2024-04-25T19:02:39.050319+0000","last_became_peered":"2024-04-25T19:02:39.050319+0000","last_unstale":"2024-04-25T19:02:45.130188+0000","last_undegraded":"2024-04-25T19:02:45.130188+0000","last_fullsized":"2024-04-25T19:02:45.130188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132040+0000","last_change":"2024-04-25T19:02:40.076010+0000","last_active":"2024-04-25T19:02:45.132040+0000","last_peered":"2024-04-25T19:02:45.132040+0000","last_clean":"2024-04-25T19:02:45.132040+0000","last_became_active":"2024-04-25T19:02:40.075677+0000","last_became_peered":"2024-04-25T19:02:40.075677+0000","last_unstale":"2024-04-25T19:02:45.132040+0000","last_undegraded":"2024-04-25T19:02:45.132040+0000","last_fullsized":"2024-04-25T19:02:45.132040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135539+0000","last_change":"2024-04-25T19:02:44.154362+0000","last_active":"2024-04-25T19:02:45.135539+0000","last_peered":"2024-04-25T19:02:45.135539+0000","last_clean":"2024-04-25T19:02:45.135539+0000","last_became_active":"2024-04-25T19:02:44.153509+0000","last_became_peered":"2024-04-25T19:02:44.153509+0000","last_unstale":"2024-04-25T19:02:45.135539+0000","last_undegraded":"2024-04-25T19:02:45.135539+0000","last_fullsized":"2024-04-25T19:02:45.135539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:16.467033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201054+0000","last_change":"2024-04-25T19:02:39.045741+0000","last_active":"2024-04-25T19:02:45.201054+0000","last_peered":"2024-04-25T19:02:45.201054+0000","last_clean":"2024-04-25T19:02:45.201054+0000","last_became_active":"2024-04-25T19:02:39.045015+0000","last_became_peered":"2024-04-25T19:02:39.045015+0000","last_unstale":"2024-04-25T19:02:45.201054+0000","last_undegraded":"2024-04-25T19:02:45.201054+0000","last_fullsized":"2024-04-25T19:02:45.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199483+0000","last_change":"2024-04-25T19:02:40.080804+0000","last_active":"2024-04-25T19:02:45.199483+0000","last_peered":"2024-04-25T19:02:45.199483+0000","last_clean":"2024-04-25T19:02:45.199483+0000","last_became_active":"2024-04-25T19:02:40.080421+0000","last_became_peered":"2024-04-25T19:02:40.080421+0000","last_unstale":"2024-04-25T19:02:45.199483+0000","last_undegraded":"2024-04-25T19:02:45.199483+0000","last_fullsized":"2024-04-25T19:02:45.199483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202500+0000","last_change":"2024-04-25T19:02:44.163067+0000","last_active":"2024-04-25T19:02:45.202500+0000","last_peered":"2024-04-25T19:02:45.202500+0000","last_clean":"2024-04-25T19:02:45.202500+0000","last_became_active":"2024-04-25T19:02:44.162000+0000","last_became_peered":"2024-04-25T19:02:44.162000+0000","last_unstale":"2024-04-25T19:02:45.202500+0000","last_undegraded":"2024-04-25T19:02:45.202500+0000","last_fullsized":"2024-04-25T19:02:45.202500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:59.430583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198599+0000","last_change":"2024-04-25T19:02:40.063088+0000","last_active":"2024-04-25T19:02:45.198599+0000","last_peered":"2024-04-25T19:02:45.198599+0000","last_clean":"2024-04-25T19:02:45.198599+0000","last_became_active":"2024-04-25T19:02:40.062379+0000","last_became_peered":"2024-04-25T19:02:40.062379+0000","last_unstale":"2024-04-25T19:02:45.198599+0000","last_undegraded":"2024-04-25T19:02:45.198599+0000","last_fullsized":"2024-04-25T19:02:45.198599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200902+0000","last_change":"2024-04-25T19:02:40.065240+0000","last_active":"2024-04-25T19:02:45.200902+0000","last_peered":"2024-04-25T19:02:45.200902+0000","last_clean":"2024-04-25T19:02:45.200902+0000","last_became_active":"2024-04-25T19:02:40.064900+0000","last_became_peered":"2024-04-25T19:02:40.064900+0000","last_unstale":"2024-04-25T19:02:45.200902+0000","last_undegraded":"2024-04-25T19:02:45.200902+0000","last_fullsized":"2024-04-25T19:02:45.200902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:34.156431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129604+0000","last_change":"2024-04-25T19:02:40.084316+0000","last_active":"2024-04-25T19:02:45.129604+0000","last_peered":"2024-04-25T19:02:45.129604+0000","last_clean":"2024-04-25T19:02:45.129604+0000","last_became_active":"2024-04-25T19:02:40.084014+0000","last_became_peered":"2024-04-25T19:02:40.084014+0000","last_unstale":"2024-04-25T19:02:45.129604+0000","last_undegraded":"2024-04-25T19:02:45.129604+0000","last_fullsized":"2024-04-25T19:02:45.129604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200778+0000","last_change":"2024-04-25T19:02:40.070357+0000","last_active":"2024-04-25T19:02:45.200778+0000","last_peered":"2024-04-25T19:02:45.200778+0000","last_clean":"2024-04-25T19:02:45.200778+0000","last_became_active":"2024-04-25T19:02:40.070028+0000","last_became_peered":"2024-04-25T19:02:40.070028+0000","last_unstale":"2024-04-25T19:02:45.200778+0000","last_undegraded":"2024-04-25T19:02:45.200778+0000","last_fullsized":"2024-04-25T19:02:45.200778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137153+0000","last_change":"2024-04-25T19:02:40.088098+0000","last_active":"2024-04-25T19:02:45.137153+0000","last_peered":"2024-04-25T19:02:45.137153+0000","last_clean":"2024-04-25T19:02:45.137153+0000","last_became_active":"2024-04-25T19:02:40.087738+0000","last_became_peered":"2024-04-25T19:02:40.087738+0000","last_unstale":"2024-04-25T19:02:45.137153+0000","last_undegraded":"2024-04-25T19:02:45.137153+0000","last_fullsized":"2024-04-25T19:02:45.137153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:08.428103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137734+0000","last_change":"2024-04-25T19:02:40.063201+0000","last_active":"2024-04-25T19:02:45.137734+0000","last_peered":"2024-04-25T19:02:45.137734+0000","last_clean":"2024-04-25T19:02:45.137734+0000","last_became_active":"2024-04-25T19:02:40.060739+0000","last_became_peered":"2024-04-25T19:02:40.060739+0000","last_unstale":"2024-04-25T19:02:45.137734+0000","last_undegraded":"2024-04-25T19:02:45.137734+0000","last_fullsized":"2024-04-25T19:02:45.137734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:40.756295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199236+0000","last_change":"2024-04-25T19:02:40.076099+0000","last_active":"2024-04-25T19:02:45.199236+0000","last_peered":"2024-04-25T19:02:45.199236+0000","last_clean":"2024-04-25T19:02:45.199236+0000","last_became_active":"2024-04-25T19:02:40.075690+0000","last_became_peered":"2024-04-25T19:02:40.075690+0000","last_unstale":"2024-04-25T19:02:45.199236+0000","last_undegraded":"2024-04-25T19:02:45.199236+0000","last_fullsized":"2024-04-25T19:02:45.199236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202228+0000","last_change":"2024-04-25T19:02:40.075377+0000","last_active":"2024-04-25T19:02:45.202228+0000","last_peered":"2024-04-25T19:02:45.202228+0000","last_clean":"2024-04-25T19:02:45.202228+0000","last_became_active":"2024-04-25T19:02:40.069488+0000","last_became_peered":"2024-04-25T19:02:40.069488+0000","last_unstale":"2024-04-25T19:02:45.202228+0000","last_undegraded":"2024-04-25T19:02:45.202228+0000","last_fullsized":"2024-04-25T19:02:45.202228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128848+0000","last_change":"2024-04-25T19:02:40.083952+0000","last_active":"2024-04-25T19:02:45.128848+0000","last_peered":"2024-04-25T19:02:45.128848+0000","last_clean":"2024-04-25T19:02:45.128848+0000","last_became_active":"2024-04-25T19:02:40.083612+0000","last_became_peered":"2024-04-25T19:02:40.083612+0000","last_unstale":"2024-04-25T19:02:45.128848+0000","last_undegraded":"2024-04-25T19:02:45.128848+0000","last_fullsized":"2024-04-25T19:02:45.128848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199584+0000","last_change":"2024-04-25T19:02:40.067612+0000","last_active":"2024-04-25T19:02:45.199584+0000","last_peered":"2024-04-25T19:02:45.199584+0000","last_clean":"2024-04-25T19:02:45.199584+0000","last_became_active":"2024-04-25T19:02:40.067294+0000","last_became_peered":"2024-04-25T19:02:40.067294+0000","last_unstale":"2024-04-25T19:02:45.199584+0000","last_undegraded":"2024-04-25T19:02:45.199584+0000","last_fullsized":"2024-04-25T19:02:45.199584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197462+0000","last_change":"2024-04-25T19:02:40.073442+0000","last_active":"2024-04-25T19:02:45.197462+0000","last_peered":"2024-04-25T19:02:45.197462+0000","last_clean":"2024-04-25T19:02:45.197462+0000","last_became_active":"2024-04-25T19:02:40.073045+0000","last_became_peered":"2024-04-25T19:02:40.073045+0000","last_unstale":"2024-04-25T19:02:45.197462+0000","last_undegraded":"2024-04-25T19:02:45.197462+0000","last_fullsized":"2024-04-25T19:02:45.197462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131761+0000","last_change":"2024-04-25T19:02:40.050494+0000","last_active":"2024-04-25T19:02:45.131761+0000","last_peered":"2024-04-25T19:02:45.131761+0000","last_clean":"2024-04-25T19:02:45.131761+0000","last_became_active":"2024-04-25T19:02:40.050067+0000","last_became_peered":"2024-04-25T19:02:40.050067+0000","last_unstale":"2024-04-25T19:02:45.131761+0000","last_undegraded":"2024-04-25T19:02:45.131761+0000","last_fullsized":"2024-04-25T19:02:45.131761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198872+0000","last_change":"2024-04-25T19:02:40.071347+0000","last_active":"2024-04-25T19:02:45.198872+0000","last_peered":"2024-04-25T19:02:45.198872+0000","last_clean":"2024-04-25T19:02:45.198872+0000","last_became_active":"2024-04-25T19:02:40.071062+0000","last_became_peered":"2024-04-25T19:02:40.071062+0000","last_unstale":"2024-04-25T19:02:45.198872+0000","last_undegraded":"2024-04-25T19:02:45.198872+0000","last_fullsized":"2024-04-25T19:02:45.198872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130821+0000","last_change":"2024-04-25T19:02:40.076710+0000","last_active":"2024-04-25T19:02:45.130821+0000","last_peered":"2024-04-25T19:02:45.130821+0000","last_clean":"2024-04-25T19:02:45.130821+0000","last_became_active":"2024-04-25T19:02:40.071132+0000","last_became_peered":"2024-04-25T19:02:40.071132+0000","last_unstale":"2024-04-25T19:02:45.130821+0000","last_undegraded":"2024-04-25T19:02:45.130821+0000","last_fullsized":"2024-04-25T19:02:45.130821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:21:56.124743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202755+0000","last_change":"2024-04-25T19:02:40.076746+0000","last_active":"2024-04-25T19:02:45.202755+0000","last_peered":"2024-04-25T19:02:45.202755+0000","last_clean":"2024-04-25T19:02:45.202755+0000","last_became_active":"2024-04-25T19:02:40.068835+0000","last_became_peered":"2024-04-25T19:02:40.068835+0000","last_unstale":"2024-04-25T19:02:45.202755+0000","last_undegraded":"2024-04-25T19:02:45.202755+0000","last_fullsized":"2024-04-25T19:02:45.202755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132119+0000","last_change":"2024-04-25T19:02:40.090029+0000","last_active":"2024-04-25T19:02:45.132119+0000","last_peered":"2024-04-25T19:02:45.132119+0000","last_clean":"2024-04-25T19:02:45.132119+0000","last_became_active":"2024-04-25T19:02:40.088886+0000","last_became_peered":"2024-04-25T19:02:40.088886+0000","last_unstale":"2024-04-25T19:02:45.132119+0000","last_undegraded":"2024-04-25T19:02:45.132119+0000","last_fullsized":"2024-04-25T19:02:45.132119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:38.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128770+0000","last_change":"2024-04-25T19:02:40.079792+0000","last_active":"2024-04-25T19:02:45.128770+0000","last_peered":"2024-04-25T19:02:45.128770+0000","last_clean":"2024-04-25T19:02:45.128770+0000","last_became_active":"2024-04-25T19:02:40.071455+0000","last_became_peered":"2024-04-25T19:02:40.071455+0000","last_unstale":"2024-04-25T19:02:45.128770+0000","last_undegraded":"2024-04-25T19:02:45.128770+0000","last_fullsized":"2024-04-25T19:02:45.128770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198995+0000","last_change":"2024-04-25T19:02:40.088240+0000","last_active":"2024-04-25T19:02:45.198995+0000","last_peered":"2024-04-25T19:02:45.198995+0000","last_clean":"2024-04-25T19:02:45.198995+0000","last_became_active":"2024-04-25T19:02:40.087892+0000","last_became_peered":"2024-04-25T19:02:40.087892+0000","last_unstale":"2024-04-25T19:02:45.198995+0000","last_undegraded":"2024-04-25T19:02:45.198995+0000","last_fullsized":"2024-04-25T19:02:45.198995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133970+0000","last_change":"2024-04-25T19:02:40.062612+0000","last_active":"2024-04-25T19:02:45.133970+0000","last_peered":"2024-04-25T19:02:45.133970+0000","last_clean":"2024-04-25T19:02:45.133970+0000","last_became_active":"2024-04-25T19:02:40.060475+0000","last_became_peered":"2024-04-25T19:02:40.060475+0000","last_unstale":"2024-04-25T19:02:45.133970+0000","last_undegraded":"2024-04-25T19:02:45.133970+0000","last_fullsized":"2024-04-25T19:02:45.133970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:10.198621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135782+0000","last_change":"2024-04-25T19:02:40.091050+0000","last_active":"2024-04-25T19:02:45.135782+0000","last_peered":"2024-04-25T19:02:45.135782+0000","last_clean":"2024-04-25T19:02:45.135782+0000","last_became_active":"2024-04-25T19:02:40.090640+0000","last_became_peered":"2024-04-25T19:02:40.090640+0000","last_unstale":"2024-04-25T19:02:45.135782+0000","last_undegraded":"2024-04-25T19:02:45.135782+0000","last_fullsized":"2024-04-25T19:02:45.135782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:57.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198750+0000","last_change":"2024-04-25T19:02:40.074894+0000","last_active":"2024-04-25T19:02:45.198750+0000","last_peered":"2024-04-25T19:02:45.198750+0000","last_clean":"2024-04-25T19:02:45.198750+0000","last_became_active":"2024-04-25T19:02:40.074142+0000","last_became_peered":"2024-04-25T19:02:40.074142+0000","last_unstale":"2024-04-25T19:02:45.198750+0000","last_undegraded":"2024-04-25T19:02:45.198750+0000","last_fullsized":"2024-04-25T19:02:45.198750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198798+0000","last_change":"2024-04-25T19:02:40.087831+0000","last_active":"2024-04-25T19:02:45.198798+0000","last_peered":"2024-04-25T19:02:45.198798+0000","last_clean":"2024-04-25T19:02:45.198798+0000","last_became_active":"2024-04-25T19:02:40.087495+0000","last_became_peered":"2024-04-25T19:02:40.087495+0000","last_unstale":"2024-04-25T19:02:45.198798+0000","last_undegraded":"2024-04-25T19:02:45.198798+0000","last_fullsized":"2024-04-25T19:02:45.198798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197993+0000","last_change":"2024-04-25T19:02:40.067092+0000","last_active":"2024-04-25T19:02:45.197993+0000","last_peered":"2024-04-25T19:02:45.197993+0000","last_clean":"2024-04-25T19:02:45.197993+0000","last_became_active":"2024-04-25T19:02:40.066752+0000","last_became_peered":"2024-04-25T19:02:40.066752+0000","last_unstale":"2024-04-25T19:02:45.197993+0000","last_undegraded":"2024-04-25T19:02:45.197993+0000","last_fullsized":"2024-04-25T19:02:45.197993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200560+0000","last_change":"2024-04-25T19:02:40.069596+0000","last_active":"2024-04-25T19:02:45.200560+0000","last_peered":"2024-04-25T19:02:45.200560+0000","last_clean":"2024-04-25T19:02:45.200560+0000","last_became_active":"2024-04-25T19:02:40.069272+0000","last_became_peered":"2024-04-25T19:02:40.069272+0000","last_unstale":"2024-04-25T19:02:45.200560+0000","last_undegraded":"2024-04-25T19:02:45.200560+0000","last_fullsized":"2024-04-25T19:02:45.200560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198925+0000","last_change":"2024-04-25T19:02:40.085542+0000","last_active":"2024-04-25T19:02:45.198925+0000","last_peered":"2024-04-25T19:02:45.198925+0000","last_clean":"2024-04-25T19:02:45.198925+0000","last_became_active":"2024-04-25T19:02:40.085203+0000","last_became_peered":"2024-04-25T19:02:40.085203+0000","last_unstale":"2024-04-25T19:02:45.198925+0000","last_undegraded":"2024-04-25T19:02:45.198925+0000","last_fullsized":"2024-04-25T19:02:45.198925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197180+0000","last_change":"2024-04-25T19:02:40.070062+0000","last_active":"2024-04-25T19:02:45.197180+0000","last_peered":"2024-04-25T19:02:45.197180+0000","last_clean":"2024-04-25T19:02:45.197180+0000","last_became_active":"2024-04-25T19:02:40.069224+0000","last_became_peered":"2024-04-25T19:02:40.069224+0000","last_unstale":"2024-04-25T19:02:45.197180+0000","last_undegraded":"2024-04-25T19:02:45.197180+0000","last_fullsized":"2024-04-25T19:02:45.197180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198254+0000","last_change":"2024-04-25T19:02:40.055714+0000","last_active":"2024-04-25T19:02:45.198254+0000","last_peered":"2024-04-25T19:02:45.198254+0000","last_clean":"2024-04-25T19:02:45.198254+0000","last_became_active":"2024-04-25T19:02:40.052418+0000","last_became_peered":"2024-04-25T19:02:40.052418+0000","last_unstale":"2024-04-25T19:02:45.198254+0000","last_undegraded":"2024-04-25T19:02:45.198254+0000","last_fullsized":"2024-04-25T19:02:45.198254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198608+0000","last_change":"2024-04-25T19:02:40.087716+0000","last_active":"2024-04-25T19:02:45.198608+0000","last_peered":"2024-04-25T19:02:45.198608+0000","last_clean":"2024-04-25T19:02:45.198608+0000","last_became_active":"2024-04-25T19:02:40.085461+0000","last_became_peered":"2024-04-25T19:02:40.085461+0000","last_unstale":"2024-04-25T19:02:45.198608+0000","last_undegraded":"2024-04-25T19:02:45.198608+0000","last_fullsized":"2024-04-25T19:02:45.198608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198153+0000","last_change":"2024-04-25T19:02:40.053676+0000","last_active":"2024-04-25T19:02:45.198153+0000","last_peered":"2024-04-25T19:02:45.198153+0000","last_clean":"2024-04-25T19:02:45.198153+0000","last_became_active":"2024-04-25T19:02:40.052518+0000","last_became_peered":"2024-04-25T19:02:40.052518+0000","last_unstale":"2024-04-25T19:02:45.198153+0000","last_undegraded":"2024-04-25T19:02:45.198153+0000","last_fullsized":"2024-04-25T19:02:45.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:54.192825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201739+0000","last_change":"2024-04-25T19:02:40.076362+0000","last_active":"2024-04-25T19:02:45.201739+0000","last_peered":"2024-04-25T19:02:45.201739+0000","last_clean":"2024-04-25T19:02:45.201739+0000","last_became_active":"2024-04-25T19:02:40.070187+0000","last_became_peered":"2024-04-25T19:02:40.070187+0000","last_unstale":"2024-04-25T19:02:45.201739+0000","last_undegraded":"2024-04-25T19:02:45.201739+0000","last_fullsized":"2024-04-25T19:02:45.201739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202327+0000","last_change":"2024-04-25T19:02:40.072359+0000","last_active":"2024-04-25T19:02:45.202327+0000","last_peered":"2024-04-25T19:02:45.202327+0000","last_clean":"2024-04-25T19:02:45.202327+0000","last_became_active":"2024-04-25T19:02:40.071678+0000","last_became_peered":"2024-04-25T19:02:40.071678+0000","last_unstale":"2024-04-25T19:02:45.202327+0000","last_undegraded":"2024-04-25T19:02:45.202327+0000","last_fullsized":"2024-04-25T19:02:45.202327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198720+0000","last_change":"2024-04-25T19:02:40.078568+0000","last_active":"2024-04-25T19:02:45.198720+0000","last_peered":"2024-04-25T19:02:45.198720+0000","last_clean":"2024-04-25T19:02:45.198720+0000","last_became_active":"2024-04-25T19:02:40.078287+0000","last_became_peered":"2024-04-25T19:02:40.078287+0000","last_unstale":"2024-04-25T19:02:45.198720+0000","last_undegraded":"2024-04-25T19:02:45.198720+0000","last_fullsized":"2024-04-25T19:02:45.198720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:05.925 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T19:03:05.926 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T19:03:05.926 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T19:03:05.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T19:03:05.926 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T19:03:06.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.237+0000 7f75ad000640 1 Processor -- start 2024-04-25T19:03:06.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.237+0000 7f75ad000640 1 -- start start 2024-04-25T19:03:06.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75ad000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75a813fbf0 0x7f75a806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75ad000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 0x7f75a8142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75ad000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a806ded0 con 0x7f75a8142a70 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75ad000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f75a80b5650 con 0x7f75a813fbf0 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75ad000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f75a80b5c70 con 0x7f75a806dac0 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75a7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75a813fbf0 0x7f75a806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75a7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f75a8142a70 0x7f75a81607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45244/0 (socket says 172.21.15.123:45244) 2024-04-25T19:03:06.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f75a7e00640 1 -- 172.21.15.123:0/3037344484 learned_addr learned my addr 172.21.15.123:0/3037344484 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.239+0000 7f759fe00640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 0x7f75a8142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f75a7400640 1 -- 172.21.15.123:0/3037344484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 msgr2=0x7f75a8142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f75a7400640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 0x7f75a8142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f75a7400640 1 -- 172.21.15.123:0/3037344484 >> v1:172.21.15.123:6789/0 conn(0x7f75a8142a70 legacy=0x7f75a81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f75a7400640 1 -- 172.21.15.123:0/3037344484 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80b5f50 con 0x7f75a813fbf0 2024-04-25T19:03:06.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f75a6a00640 1 -- 172.21.15.123:0/3037344484 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058222168 0 0) 0x7f75a0001490 con 0x7f75a8142a70 2024-04-25T19:03:06.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.240+0000 7f759fe00640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 0x7f75a8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:06.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.241+0000 7f75a7400640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a806d580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f75980097a0 tx=0x7f759802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3c3566d6bc8d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.241+0000 7f75a6a00640 1 -- 172.21.15.123:0/3037344484 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598037070 con 0x7f75a813fbf0 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.241+0000 7f75a6a00640 1 -- 172.21.15.123:0/3037344484 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7598032440 con 0x7f75a813fbf0 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.241+0000 7f75a6a00640 1 -- 172.21.15.123:0/3037344484 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598032810 con 0x7f75a813fbf0 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 -- 172.21.15.123:0/3037344484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 msgr2=0x7f75a806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a806d580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f75980097a0 tx=0x7f759802a180 comp rx=0 tx=0).stop 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 -- 172.21.15.123:0/3037344484 shutdown_connections 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a806dac0 0x7f75a8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 --2- 172.21.15.123:0/3037344484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a806d580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 -- 172.21.15.123:0/3037344484 >> 172.21.15.123:0/3037344484 conn(0x7f75a8073200 msgr2=0x7f75a8073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 -- 172.21.15.123:0/3037344484 shutdown_connections 2024-04-25T19:03:06.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.242+0000 7f75ad000640 1 -- 172.21.15.123:0/3037344484 wait complete. 2024-04-25T19:03:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.244+0000 7f75ad000640 1 Processor -- start 2024-04-25T19:03:06.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.244+0000 7f75ad000640 1 -- start start 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a806dac0 0x7f75a8139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a8142a70 0x7f75a81415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80b6a30 con 0x7f75a813fbf0 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80b6510 con 0x7f75a806dac0 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75ad000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f75a8142e80 con 0x7f75a8142a70 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f759fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f759fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34146/0 (socket says 172.21.15.123:34146) 2024-04-25T19:03:06.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f759fe00640 1 -- 172.21.15.123:0/2492991634 learned_addr learned my addr 172.21.15.123:0/2492991634 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.246+0000 7f75a7e00640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a8142a70 0x7f75a81415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f75a7400640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a806dac0 0x7f75a8139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 -- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a8142a70 msgr2=0x7f75a81415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a8142a70 0x7f75a81415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 -- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a806dac0 msgr2=0x7f75a8139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a806dac0 0x7f75a8139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 -- 172.21.15.123:0/2492991634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a813ac90 con 0x7f75a813fbf0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759fe00640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f759000c7f0 tx=0x7f759000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590015070 con 0x7f75a813fbf0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7590007820 con 0x7f75a813fbf0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590007bd0 con 0x7f75a813fbf0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.247+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a813a6d0 con 0x7f75a813fbf0 2024-04-25T19:03:06.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.248+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a81542c0 con 0x7f75a813fbf0 2024-04-25T19:03:06.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.249+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8050740 con 0x7f75a813fbf0 2024-04-25T19:03:06.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.250+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7590018070 con 0x7f75a813fbf0 2024-04-25T19:03:06.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.251+0000 7f759f400640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 0x7f75840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.251+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7590095750 con 0x7f75a813fbf0 2024-04-25T19:03:06.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.251+0000 7f75a7400640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 0x7f75840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.252+0000 7f75a7400640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 0x7f75840769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7598034000 tx=0x7f759802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.256+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f759005fb00 con 0x7f75a813fbf0 2024-04-25T19:03:06.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.510+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f75a80cfb70 con 0x7f75a813fbf0 2024-04-25T19:03:06.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.520+0000 7f759f400640 1 -- 172.21.15.123:0/2492991634 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186750 (secure 0 0 0) 0x7f759005f400 con 0x7f75a813fbf0 2024-04-25T19:03:06.524 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 msgr2=0x7f75840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 0x7f75840769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7598034000 tx=0x7f759802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 msgr2=0x7f75a8152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f759000c7f0 tx=0x7f759000ccb0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 shutdown_connections 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f75840744b0 0x7f75840769c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75a8142a70 0x7f75a81415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75a813fbf0 0x7f75a8152c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 --2- 172.21.15.123:0/2492991634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75a806dac0 0x7f75a8139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 >> 172.21.15.123:0/2492991634 conn(0x7f75a8073200 msgr2=0x7f75a8073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.524+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 shutdown_connections 2024-04-25T19:03:06.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.525+0000 7f75ad000640 1 -- 172.21.15.123:0/2492991634 wait complete. 2024-04-25T19:03:06.554 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1011550908},{"type":"v1","addr":"172.21.15.123:6833","nonce":1011550908}]},"active_addr":"172.21.15.123:6833/1011550908","active_change":"2024-04-25T19:02:18.933873+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3604958439}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3607111911}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3271854695}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2497662247}]}]} 2024-04-25T19:03:06.558 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T19:03:06.559 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T19:03:06.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.819+0000 7f9522c00640 1 Processor -- start 2024-04-25T19:03:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.819+0000 7f9522c00640 1 -- start start 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 0x7f951c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f951c06d580 0x7f951c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f951c06ded0 con 0x7f951c06d580 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f951c15f070 con 0x7f951c06fe20 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.821+0000 7f9522c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f951c06f340 con 0x7f951c15e2f0 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f951c06d580 0x7f951c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f951c06d580 0x7f951c06d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34156/0 (socket says 172.21.15.123:34156) 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 -- 172.21.15.123:0/4103136156 learned_addr learned my addr 172.21.15.123:0/4103136156 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9521800640 1 --2- 172.21.15.123:0/4103136156 >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 0x7f951c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 -- 172.21.15.123:0/4103136156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 msgr2=0x7f951c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9522200640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c15e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 -- 172.21.15.123:0/4103136156 >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 msgr2=0x7f951c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 --2- 172.21.15.123:0/4103136156 >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 0x7f951c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9520e00640 1 -- 172.21.15.123:0/4103136156 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c06f620 con 0x7f951c06d580 2024-04-25T19:03:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.822+0000 7f9521800640 1 --2- 172.21.15.123:0/4103136156 >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 0x7f951c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.823+0000 7f9520e00640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f951c074310 tx=0x7f950802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e50f992934d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.823+0000 7f951be00640 1 -- 172.21.15.123:0/4103136156 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508037070 con 0x7f951c06d580 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f951be00640 1 -- 172.21.15.123:0/4103136156 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9508032440 con 0x7f951c06d580 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f951be00640 1 -- 172.21.15.123:0/4103136156 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508032810 con 0x7f951c06d580 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 -- 172.21.15.123:0/4103136156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 msgr2=0x7f951c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f951c074310 tx=0x7f950802a140 comp rx=0 tx=0).stop 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 -- 172.21.15.123:0/4103136156 shutdown_connections 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 --2- 172.21.15.123:0/4103136156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c06d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 --2- 172.21.15.123:0/4103136156 >> v2:172.21.15.146:3300/0 conn(0x7f951c06fe20 0x7f951c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 -- 172.21.15.123:0/4103136156 >> 172.21.15.123:0/4103136156 conn(0x7f951c073200 msgr2=0x7f951c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 -- 172.21.15.123:0/4103136156 shutdown_connections 2024-04-25T19:03:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.824+0000 7f9522c00640 1 -- 172.21.15.123:0/4103136156 wait complete. 2024-04-25T19:03:06.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.827+0000 7f9522c00640 1 Processor -- start 2024-04-25T19:03:06.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.827+0000 7f9522c00640 1 -- start start 2024-04-25T19:03:06.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c06fe20 0x7f951c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c1f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15ec40 con 0x7f951c06d580 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15bbf0 con 0x7f951c06fe20 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.828+0000 7f9522c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f951c15edc0 con 0x7f951c15e2f0 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34162/0 (socket says 172.21.15.123:34162) 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 -- 172.21.15.123:0/3930733221 learned_addr learned my addr 172.21.15.123:0/3930733221 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 -- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 msgr2=0x7f951c1f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c1f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9520e00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c06fe20 0x7f951c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 -- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c06fe20 msgr2=0x7f951c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c06fe20 0x7f951c0b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 -- 172.21.15.123:0/3930733221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c0b56d0 con 0x7f951c06d580 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9521800640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f951000cde0 tx=0x7f951000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.829+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000f680 con 0x7f951c06d580 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.830+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951000f820 con 0x7f951c06d580 2024-04-25T19:03:06.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.830+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c0cac10 con 0x7f951c06d580 2024-04-25T19:03:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.830+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000fc10 con 0x7f951c06d580 2024-04-25T19:03:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.830+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c0b4990 con 0x7f951c06d580 2024-04-25T19:03:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.831+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c070a00 con 0x7f951c06d580 2024-04-25T19:03:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.832+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9510018400 con 0x7f951c06d580 2024-04-25T19:03:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.833+0000 7f9519600640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 0x7f94fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.833+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9510096360 con 0x7f951c06d580 2024-04-25T19:03:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.833+0000 7f9520e00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 0x7f94fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.834+0000 7f9520e00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 0x7f94fc0769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f950802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:06.837+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9510014030 con 0x7f951c06d580 2024-04-25T19:03:07.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.038+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f951c073790 con 0x7f951c06d580 2024-04-25T19:03:07.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.040+0000 7f9519600640 1 -- 172.21.15.123:0/3930733221 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f9510060160 con 0x7f951c06d580 2024-04-25T19:03:07.042 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:07.043 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 msgr2=0x7f94fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 0x7f94fc0769f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f950802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 msgr2=0x7f951c0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f951000cde0 tx=0x7f951000a650 comp rx=0 tx=0).stop 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 shutdown_connections 2024-04-25T19:03:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94fc0744e0 0x7f94fc0769f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15e2f0 0x7f951c1f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c06fe20 0x7f951c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 --2- 172.21.15.123:0/3930733221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c06d580 0x7f951c0b6290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 >> 172.21.15.123:0/3930733221 conn(0x7f951c073200 msgr2=0x7f951c15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.043+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 shutdown_connections 2024-04-25T19:03:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.044+0000 7f9522c00640 1 -- 172.21.15.123:0/3930733221 wait complete. 2024-04-25T19:03:07.071 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T19:03:07.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:07.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.337+0000 7f8cd0a00640 1 Processor -- start 2024-04-25T19:03:07.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.337+0000 7f8cd0a00640 1 -- start start 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cd0a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ccc0b57f0 0x7f8ccc0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cd0a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ccc0b6140 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cd0a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ccc0b4750 con 0x7f8ccc15df00 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cd0a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc15e310 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cc9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ccc0b57f0 0x7f8ccc0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cca800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8ccc0ae7e0 0x7f8ccc0aebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45254/0 (socket says 172.21.15.123:45254) 2024-04-25T19:03:07.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.339+0000 7f8cca800640 1 -- 172.21.15.123:0/4171715798 learned_addr learned my addr 172.21.15.123:0/4171715798 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:07.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.340+0000 7f8cc9400640 1 -- 172.21.15.123:0/4171715798 >> v1:172.21.15.146:6789/0 conn(0x7f8ccc15df00 legacy=0x7f8ccc06f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.340+0000 7f8cc9400640 1 -- 172.21.15.123:0/4171715798 >> v1:172.21.15.123:6789/0 conn(0x7f8ccc0ae7e0 legacy=0x7f8ccc0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.340+0000 7f8cc9400640 1 -- 172.21.15.123:0/4171715798 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc0b4a30 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.341+0000 7f8cc9400640 1 --2- 172.21.15.123:0/4171715798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0b57f0 0x7f8ccc0b5c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc374707315feb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:07.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.341+0000 7f8cc8a00640 1 -- 172.21.15.123:0/4171715798 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007740 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.341+0000 7f8cc8a00640 1 -- 172.21.15.123:0/4171715798 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb80078e0 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.341+0000 7f8cc8a00640 1 -- 172.21.15.123:0/4171715798 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007cb0 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 -- 172.21.15.123:0/4171715798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0b57f0 msgr2=0x7f8ccc0b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/4171715798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0b57f0 0x7f8ccc0b5c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb802a660 comp rx=0 tx=0).stop 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 -- 172.21.15.123:0/4171715798 shutdown_connections 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/4171715798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0b57f0 0x7f8ccc0b5c00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 -- 172.21.15.123:0/4171715798 >> 172.21.15.123:0/4171715798 conn(0x7f8ccc0d1ae0 msgr2=0x7f8ccc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 -- 172.21.15.123:0/4171715798 shutdown_connections 2024-04-25T19:03:07.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.342+0000 7f8cd0a00640 1 -- 172.21.15.123:0/4171715798 wait complete. 2024-04-25T19:03:07.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.345+0000 7f8cd0a00640 1 Processor -- start 2024-04-25T19:03:07.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.345+0000 7f8cd0a00640 1 -- start start 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ccc0b57f0 0x7f8ccc156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ccc15df00 0x7f8ccc0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b4ff0 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b5360 con 0x7f8ccc15df00 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.346+0000 7f8cd0a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc06f950 con 0x7f8ccc0b57f0 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34184/0 (socket says 172.21.15.123:34184) 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 -- 172.21.15.123:0/1211642478 learned_addr learned my addr 172.21.15.123:0/1211642478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:07.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9400640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ccc0b57f0 0x7f8ccc156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cca800640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ccc15df00 0x7f8ccc0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 -- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ccc0b57f0 msgr2=0x7f8ccc156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ccc0b57f0 0x7f8ccc156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 -- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ccc15df00 msgr2=0x7f8ccc0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ccc15df00 0x7f8ccc0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 -- 172.21.15.123:0/1211642478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc0caee0 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.347+0000 7f8cc9e00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c400 tx=0x7f8cbc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.348+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc017070 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.348+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0052d0 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.348+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc159560 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.348+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc003830 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.348+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc159b20 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.349+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc076a40 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.350+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8cbc01a070 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.351+0000 7f8cc2000640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 0x7f8cac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:07.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.351+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8cbc0948e0 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.351+0000 7f8cc9400640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 0x7f8cac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:07.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.352+0000 7f8cc9400640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 0x7f8cac0769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8034000 tx=0x7f8cb8033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:07.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.356+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cbc05ed30 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.559+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ccc0aed20 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.562+0000 7f8cc2000640 1 -- 172.21.15.123:0/1211642478 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f8cbc05e630 con 0x7f8ccc0ae7e0 2024-04-25T19:03:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:07.565 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 msgr2=0x7f8cac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 0x7f8cac0769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8034000 tx=0x7f8cb8033040 comp rx=0 tx=0).stop 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 msgr2=0x7f8ccc0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c400 tx=0x7f8cbc00c8c0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 shutdown_connections 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8cac0744e0 0x7f8cac0769f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ccc15df00 0x7f8ccc0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ccc0b57f0 0x7f8ccc156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 --2- 172.21.15.123:0/1211642478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ccc0ae7e0 0x7f8ccc0ca9a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 >> 172.21.15.123:0/1211642478 conn(0x7f8ccc0d1ae0 msgr2=0x7f8ccc0739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:07.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 shutdown_connections 2024-04-25T19:03:07.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.565+0000 7f8cd0a00640 1 -- 172.21.15.123:0/1211642478 wait complete. 2024-04-25T19:03:07.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T19:03:07.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T19:03:07.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T19:03:07.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T19:03:07.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T19:03:07.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T19:03:07.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T19:03:07.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T19:03:07.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.997+0000 7f5640e00640 1 Processor -- start 2024-04-25T19:03:07.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.997+0000 7f5640e00640 1 -- start start 2024-04-25T19:03:08.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:07.999+0000 7f83bbe00640 1 Processor -- start 2024-04-25T19:03:08.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f5640e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c06f050 0x7f563c06f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f5640e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f563c0b7440 0x7f563c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f5640e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563c0d06c0 con 0x7f563c0d02b0 2024-04-25T19:03:08.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f5640e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f563c0d2fc0 con 0x7f563c0b7440 2024-04-25T19:03:08.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f5640e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f563c0b7d90 con 0x7f563c06f050 2024-04-25T19:03:08.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.000+0000 7f83bbe00640 1 -- start start 2024-04-25T19:03:08.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.001+0000 7f563be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f563c0b7440 0x7f563c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.001+0000 7f563be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f563c0b7440 0x7f563c0b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48240/0 (socket says 172.21.15.123:48240) 2024-04-25T19:03:08.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563be00640 1 -- 172.21.15.123:0/4135618107 learned_addr learned my addr 172.21.15.123:0/4135618107 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563aa00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c06f050 0x7f563c06f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563be00640 1 -- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c06f050 msgr2=0x7f563c06f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563be00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c06f050 0x7f563c06f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563be00640 1 -- 172.21.15.123:0/4135618107 >> v1:172.21.15.146:6789/0 conn(0x7f563c0d02b0 legacy=0x7f563c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.002+0000 7f563be00640 1 -- 172.21.15.123:0/4135618107 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c0d23b0 con 0x7f563c0b7440 2024-04-25T19:03:08.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.003+0000 7f83bbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 0x7f83bc0b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.003+0000 7f83bbe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83bc15d070 0x7f83bc0b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.003+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc0b2010 0x7f83bc0d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.003+0000 7f73a5c00640 1 Processor -- start 2024-04-25T19:03:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83bbe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f83bc161930 con 0x7f83bc161400 2024-04-25T19:03:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83b3e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83bc15d070 0x7f83bc0b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f73a5c00640 1 -- start start 2024-04-25T19:03:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83bbe00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f83bc0cf970 con 0x7f83bc15d070 2024-04-25T19:03:08.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f563be00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c0b7440 0x7f563c0b7850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57b4ced8e048c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83b3e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83bc15d070 0x7f83bc0b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48252/0 (socket says 172.21.15.123:48252) 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83b3e00640 1 -- 172.21.15.123:0/2771443011 learned_addr learned my addr 172.21.15.123:0/2771443011 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83baa00640 1 --2- 172.21.15.123:0/2771443011 >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 0x7f83bc0b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.004+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc0cfc50 con 0x7f83bc0b2010 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83bb400640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc0b2010 0x7f83bc0d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f563a000640 1 -- 172.21.15.123:0/4135618107 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634036070 con 0x7f563c0b7440 2024-04-25T19:03:08.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f563a000640 1 -- 172.21.15.123:0/4135618107 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5634031600 con 0x7f563c0b7440 2024-04-25T19:03:08.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.007+0000 7f563a000640 1 -- 172.21.15.123:0/4135618107 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340319d0 con 0x7f563c0b7440 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83b3e00640 1 -- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc0b2010 msgr2=0x7f83bc0d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83b3e00640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc0b2010 0x7f83bc0d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83b3e00640 1 -- 172.21.15.123:0/2771443011 >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 msgr2=0x7f83bc0b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83b3e00640 1 --2- 172.21.15.123:0/2771443011 >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 0x7f83bc0b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83b3e00640 1 -- 172.21.15.123:0/2771443011 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc0cff30 con 0x7f83bc15d070 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.005+0000 7f83baa00640 1 --2- 172.21.15.123:0/2771443011 >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 0x7f83bc0b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:08.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.007+0000 7f83b3e00640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc15d070 0x7f83bc0b7980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0d3740 tx=0x7f83a402a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1f75b0f853f684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 -- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c0b7440 msgr2=0x7f563c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c0b7440 0x7f563c0b7850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563402a710 comp rx=0 tx=0).stop 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 -- 172.21.15.123:0/4135618107 shutdown_connections 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c0b7440 0x7f563c0b7850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 --2- 172.21.15.123:0/4135618107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c06f050 0x7f563c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 -- 172.21.15.123:0/4135618107 >> 172.21.15.123:0/4135618107 conn(0x7f563c09b4a0 msgr2=0x7f563c09b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 -- 172.21.15.123:0/4135618107 shutdown_connections 2024-04-25T19:03:08.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f9de5200640 1 Processor -- start 2024-04-25T19:03:08.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.008+0000 7f5640e00640 1 -- 172.21.15.123:0/4135618107 wait complete. 2024-04-25T19:03:08.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f83ba000640 1 -- 172.21.15.123:0/2771443011 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4037070 con 0x7f83bc15d070 2024-04-25T19:03:08.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f83ba000640 1 -- 172.21.15.123:0/2771443011 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a402ae30 con 0x7f83bc15d070 2024-04-25T19:03:08.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f83ba000640 1 -- 172.21.15.123:0/2771443011 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4032640 con 0x7f83bc15d070 2024-04-25T19:03:08.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f9de5200640 1 -- start start 2024-04-25T19:03:08.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f73a5c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f73a5c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7398048330 0x7f7398033980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f73a5c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7398006160 con 0x7f7398033ec0 2024-04-25T19:03:08.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f73a5c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f739804f6c0 con 0x7f7398048330 2024-04-25T19:03:08.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.009+0000 7f73a5c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73980342d0 con 0x7f7398047de0 2024-04-25T19:03:08.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7f739ea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc15d070 msgr2=0x7f83bc0b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7f83bbe00640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc15d070 0x7f83bc0b7980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0d3740 tx=0x7f83a402a0e0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7f739ea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47438/0 (socket says 172.21.15.123:47438) 2024-04-25T19:03:08.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7f739ea00640 1 -- 172.21.15.123:0/333622569 learned_addr learned my addr 172.21.15.123:0/333622569 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.010+0000 7fe1a5a00640 1 Processor -- start 2024-04-25T19:03:08.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f739ea00640 1 -- 172.21.15.123:0/333622569 >> v2:172.21.15.146:3300/0 conn(0x7f7398048330 msgr2=0x7f7398033980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f739ea00640 1 --2- 172.21.15.123:0/333622569 >> v2:172.21.15.146:3300/0 conn(0x7f7398048330 0x7f7398033980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f739ea00640 1 -- 172.21.15.123:0/333622569 >> v1:172.21.15.146:6789/0 conn(0x7f7398033ec0 legacy=0x7f7398053210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f739ea00640 1 -- 172.21.15.123:0/333622569 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739804ffd0 con 0x7f7398047de0 2024-04-25T19:03:08.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f5640e00640 1 Processor -- start 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f5640e00640 1 -- start start 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f9de5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 0x7f9dd8057bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7f9de5200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9dd8058b90 0x7f9dd80380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9de5200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd8038620 con 0x7f9dd80073d0 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9de5200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd8057730 con 0x7f9dd80e3a90 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9de5200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd8056e00 con 0x7f9dd8058b90 2024-04-25T19:03:08.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9ddf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 0x7f9dd8057bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9ddf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 0x7f9dd8057bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34216/0 (socket says 172.21.15.123:34216) 2024-04-25T19:03:08.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9ddf400640 1 -- 172.21.15.123:0/2626467247 learned_addr learned my addr 172.21.15.123:0/2626467247 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.011+0000 7fe1a5a00640 1 -- start start 2024-04-25T19:03:08.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f1da8800640 1 Processor -- start 2024-04-25T19:03:08.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f9ddea00640 1 --2- 172.21.15.123:0/2626467247 >> v2:172.21.15.146:3300/0 conn(0x7f9dd8058b90 0x7f9dd80380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 shutdown_connections 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f83bbe00640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc0b2010 0x7f83bc0d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f83bbe00640 1 --2- 172.21.15.123:0/2771443011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc15d070 0x7f83bc0b7980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f83bbe00640 1 --2- 172.21.15.123:0/2771443011 >> v2:172.21.15.123:3300/0 conn(0x7f83bc161400 0x7f83bc0b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 >> 172.21.15.123:0/2771443011 conn(0x7f83bc09b4a0 msgr2=0x7f83bc09b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f1da8800640 1 -- start start 2024-04-25T19:03:08.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.012+0000 7f739ea00640 1 --2- 172.21.15.123:0/333622569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7390009870 tx=0x7f739002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36a635f2d9e11de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 shutdown_connections 2024-04-25T19:03:08.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f83bbe00640 1 -- 172.21.15.123:0/2771443011 wait complete. 2024-04-25T19:03:08.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f739d600640 1 -- 172.21.15.123:0/333622569 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390037070 con 0x7f7398047de0 2024-04-25T19:03:08.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f739d600640 1 -- 172.21.15.123:0/333622569 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7390032440 con 0x7f7398047de0 2024-04-25T19:03:08.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f739d600640 1 -- 172.21.15.123:0/333622569 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390037070 con 0x7f7398047de0 2024-04-25T19:03:08.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7fe1a5a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1a00d0da0 0x7fe1a00d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7f5640e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c06f050 0x7f563c09fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7f5640e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 0x7f563c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7f5640e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f563c0d02b0 0x7f563c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7fe1a5a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a006c570 con 0x7fe1a0163ae0 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7fe1a5a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a00d1510 con 0x7fe1a00d0da0 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.014+0000 7fe1a5a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a00d4720 con 0x7fe1a006da60 2024-04-25T19:03:08.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f9ddea00640 1 -- 172.21.15.123:0/2626467247 >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 msgr2=0x7f9dd8057bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f9ddea00640 1 --2- 172.21.15.123:0/2626467247 >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 0x7f9dd8057bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f9ddea00640 1 -- 172.21.15.123:0/2626467247 >> v1:172.21.15.123:6789/0 conn(0x7f9dd80073d0 legacy=0x7f9dd80077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.013+0000 7f9ddea00640 1 -- 172.21.15.123:0/2626467247 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd80570e0 con 0x7f9dd8058b90 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f5640e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0d2c50 con 0x7f563c0d02b0 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 -- 172.21.15.123:0/333622569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 msgr2=0x7f73980e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 --2- 172.21.15.123:0/333622569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7390009870 tx=0x7f739002a270 comp rx=0 tx=0).stop 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f5640e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0d2970 con 0x7f563c06f050 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f5640e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f563c0d3450 con 0x7f563c0b7440 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f563be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f563c0d02b0 0x7f563c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f563aa00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 0x7f563c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f563b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c06f050 0x7f563c09fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f563aa00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 0x7f563c0b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47454/0 (socket says 172.21.15.123:47454) 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f563aa00640 1 -- 172.21.15.123:0/4184441548 learned_addr learned my addr 172.21.15.123:0/4184441548 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 -- 172.21.15.123:0/333622569 shutdown_connections 2024-04-25T19:03:08.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7fe19f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1a00d0da0 0x7fe1a00d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 --2- 172.21.15.123:0/333622569 >> v2:172.21.15.146:3300/0 conn(0x7f7398048330 0x7f7398033980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 --2- 172.21.15.123:0/333622569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f73980e7d80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 -- 172.21.15.123:0/333622569 >> 172.21.15.123:0/333622569 conn(0x7f7398054d20 msgr2=0x7f7398055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f9ddea00640 1 --2- 172.21.15.123:0/2626467247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd8058b90 0x7f9dd80380e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009ee0 tx=0x7f9dcc02a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1728ca0c94de56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7fe1a5000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7fe1a5000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47464/0 (socket says 172.21.15.123:47464) 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7fe1a5000640 1 -- 172.21.15.123:0/278523029 learned_addr learned my addr 172.21.15.123:0/278523029 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.015+0000 7f73a5c00640 1 -- 172.21.15.123:0/333622569 shutdown_connections 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f9dde000640 1 -- 172.21.15.123:0/2626467247 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc037070 con 0x7f9dd8058b90 2024-04-25T19:03:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f1da8800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1da0160070 0x7f1da00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f1da8800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da00cc880 con 0x7f1da0070850 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f1da8800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da00ccb60 con 0x7f1da00d0750 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f1da8800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1da00cd490 con 0x7f1da0160070 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f73a5c00640 1 -- 172.21.15.123:0/333622569 wait complete. 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f9dde000640 1 -- 172.21.15.123:0/2626467247 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc02ad40 con 0x7f9dd8058b90 2024-04-25T19:03:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f9dde000640 1 -- 172.21.15.123:0/2626467247 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc032660 con 0x7f9dd8058b90 2024-04-25T19:03:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7f1da6800640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1da00d0750 0x7f1da006c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:47462/0 (socket says 172.21.15.123:47462) 2024-04-25T19:03:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7f1da6800640 1 -- 172.21.15.123:0/2164041811 learned_addr learned my addr 172.21.15.123:0/2164041811 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7fe1a5000640 1 -- 172.21.15.123:0/278523029 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00d0da0 msgr2=0x7fe1a00d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7fe1a5000640 1 --2- 172.21.15.123:0/278523029 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00d0da0 0x7fe1a00d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7fe1a5000640 1 -- 172.21.15.123:0/278523029 >> v1:172.21.15.123:6789/0 conn(0x7fe1a0163ae0 legacy=0x7fe1a00d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f563b400640 1 -- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 msgr2=0x7f563c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f563b400640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 0x7f563c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f563b400640 1 -- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f563c0d02b0 msgr2=0x7f563c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f563b400640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f563c0d02b0 0x7f563c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7fe1a5000640 1 -- 172.21.15.123:0/278523029 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a00d4950 con 0x7fe1a006da60 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.016+0000 7f563b400640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c0b0710 con 0x7f563c06f050 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.017+0000 7f563b400640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c06f050 0x7f563c09fde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f563c0d0890 tx=0x7f562c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7f563be00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f563c0d02b0 0x7f563c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7f563aa00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f563c0b7440 0x7f563c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7fee19400640 1 Processor -- start 2024-04-25T19:03:08.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7fee19400640 1 -- start start 2024-04-25T19:03:08.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7f9de5200640 1 -- 172.21.15.123:0/2626467247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd8058b90 msgr2=0x7f9dd80380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7f9de5200640 1 --2- 172.21.15.123:0/2626467247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd8058b90 0x7f9dd80380e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009ee0 tx=0x7f9dcc02a030 comp rx=0 tx=0).stop 2024-04-25T19:03:08.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7fe1a5000640 1 --2- 172.21.15.123:0/278523029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe198009e50 tx=0x7fe19802a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fd907a443cdc69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.018+0000 7f1da5e00640 1 --2- 172.21.15.123:0/2164041811 >> v2:172.21.15.123:3300/0 conn(0x7f1da0160070 0x7f1da00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7fc799800640 1 Processor -- start 2024-04-25T19:03:08.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c017070 con 0x7f563c06f050 2024-04-25T19:03:08.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f1da4a00640 1 -- 172.21.15.123:0/2164041811 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359427847 0 0) 0x7f1da00ccb60 con 0x7f1da00d0750 2024-04-25T19:03:08.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f1da4a00640 1 -- 172.21.15.123:0/2164041811 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d88003520 con 0x7f1da00d0750 2024-04-25T19:03:08.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c0052d0 con 0x7f563c06f050 2024-04-25T19:03:08.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c011850 con 0x7f563c06f050 2024-04-25T19:03:08.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7fc799800640 1 -- start start 2024-04-25T19:03:08.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7fe19ea00640 1 -- 172.21.15.123:0/278523029 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198036070 con 0x7fe1a006da60 2024-04-25T19:03:08.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7fe19ea00640 1 -- 172.21.15.123:0/278523029 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe198031600 con 0x7fe1a006da60 2024-04-25T19:03:08.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7fe19ea00640 1 -- 172.21.15.123:0/278523029 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1980319b0 con 0x7fe1a006da60 2024-04-25T19:03:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7f9de5200640 1 -- 172.21.15.123:0/2626467247 shutdown_connections 2024-04-25T19:03:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7f9de5200640 1 --2- 172.21.15.123:0/2626467247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd8058b90 0x7f9dd80380e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7f9de5200640 1 --2- 172.21.15.123:0/2626467247 >> v2:172.21.15.123:3300/0 conn(0x7f9dd80e3a90 0x7f9dd8057bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c0b09f0 con 0x7f563c06f050 2024-04-25T19:03:08.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7f9de5200640 1 -- 172.21.15.123:0/2626467247 >> 172.21.15.123:0/2626467247 conn(0x7f9dd803abf0 msgr2=0x7f9dd803b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 -- 172.21.15.123:0/278523029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 msgr2=0x7fe1a01600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/278523029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe198009e50 tx=0x7fe19802a730 comp rx=0 tx=0).stop 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f1da5e00640 1 -- 172.21.15.123:0/2164041811 >> v1:172.21.15.146:6789/0 conn(0x7f1da00d0750 legacy=0x7f1da006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f1da5e00640 1 -- 172.21.15.123:0/2164041811 >> v1:172.21.15.123:6789/0 conn(0x7f1da0070850 legacy=0x7f1da00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.019+0000 7f1da5e00640 1 -- 172.21.15.123:0/2164041811 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da00cce40 con 0x7f1da0160070 2024-04-25T19:03:08.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c0b0fb0 con 0x7f563c06f050 2024-04-25T19:03:08.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 -- 172.21.15.123:0/278523029 shutdown_connections 2024-04-25T19:03:08.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/278523029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a006da60 0x7fe1a01600b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/278523029 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00d0da0 0x7fe1a00d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.020+0000 7fe1a5a00640 1 -- 172.21.15.123:0/278523029 >> 172.21.15.123:0/278523029 conn(0x7fe1a00d6510 msgr2=0x7fe1a00b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f73a5c00640 1 Processor -- start 2024-04-25T19:03:08.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f83bbe00640 1 Processor -- start 2024-04-25T19:03:08.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f9de5200640 1 -- 172.21.15.123:0/2626467247 shutdown_connections 2024-04-25T19:03:08.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f73a5c00640 1 -- start start 2024-04-25T19:03:08.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f1da5e00640 1 --2- 172.21.15.123:0/2164041811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da00cd920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d940097d0 tx=0x7f1d9402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b67e6413706d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fe1a5a00640 1 -- 172.21.15.123:0/278523029 shutdown_connections 2024-04-25T19:03:08.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fe1a5a00640 1 -- 172.21.15.123:0/278523029 wait complete. 2024-04-25T19:03:08.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 0x7fee1406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee14163d00 0x7fee140b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee140d0a90 0x7fee140d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee140d13e0 con 0x7fee1406dca0 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fee1406cc00 con 0x7fee140d0a90 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee19400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee1406cee0 con 0x7fee14163d00 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fc799800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc79406c570 0x7fc79406daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fc799800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7940d48b0 0x7fc7940d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fc799800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc794160740 con 0x7fc794160330 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fc799800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc7940d5200 con 0x7fc79406c570 2024-04-25T19:03:08.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fc799800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc79406ca50 con 0x7fc7940d48b0 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7f1da4a00640 1 -- 172.21.15.123:0/2164041811 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94037070 con 0x7f1da0160070 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7f1da4a00640 1 -- 172.21.15.123:0/2164041811 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9402ad10 con 0x7f1da0160070 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7f1da4a00640 1 -- 172.21.15.123:0/2164041811 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94032660 con 0x7f1da0160070 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee18a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee140d0a90 0x7fee140d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee18a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee140d0a90 0x7fee140d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48284/0 (socket says 172.21.15.123:48284) 2024-04-25T19:03:08.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.022+0000 7fee18a00640 1 -- 172.21.15.123:0/3029527351 learned_addr learned my addr 172.21.15.123:0/3029527351 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee13400640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee14163d00 0x7fee140b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7f83bbe00640 1 -- start start 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee13e00640 1 --2- 172.21.15.123:0/3029527351 >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 0x7fee1406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee18a00640 1 -- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee14163d00 msgr2=0x7fee140b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee18a00640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee14163d00 0x7fee140b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee18a00640 1 -- 172.21.15.123:0/3029527351 >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 msgr2=0x7fee1406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee18a00640 1 --2- 172.21.15.123:0/3029527351 >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 0x7fee1406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fc78a000640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc794160330 0x7fc7940d3f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:47478/0 (socket says 172.21.15.123:47478) 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fc78a000640 1 -- 172.21.15.123:0/1637566531 learned_addr learned my addr 172.21.15.123:0/1637566531 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792a00640 1 --2- 172.21.15.123:0/1637566531 >> v2:172.21.15.146:3300/0 conn(0x7fc79406c570 0x7fc79406daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7f1da8800640 1 -- 172.21.15.123:0/2164041811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 msgr2=0x7f1da00cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7f1da8800640 1 --2- 172.21.15.123:0/2164041811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da00cd920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d940097d0 tx=0x7f1d9402a080 comp rx=0 tx=0).stop 2024-04-25T19:03:08.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7f1da8800640 1 -- 172.21.15.123:0/2164041811 shutdown_connections 2024-04-25T19:03:08.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc793400640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7940d48b0 0x7fc7940d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792000640 1 -- 172.21.15.123:0/1637566531 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567509327 0 0) 0x7fc794160740 con 0x7fc794160330 2024-04-25T19:03:08.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792000640 1 -- 172.21.15.123:0/1637566531 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc778003520 con 0x7fc794160330 2024-04-25T19:03:08.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7f1da8800640 1 --2- 172.21.15.123:0/2164041811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da00cd920 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7f1da8800640 1 -- 172.21.15.123:0/2164041811 >> 172.21.15.123:0/2164041811 conn(0x7f1da00ce1b0 msgr2=0x7f1da00ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.021+0000 7f9de5200640 1 -- 172.21.15.123:0/2626467247 wait complete. 2024-04-25T19:03:08.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.025+0000 7f9de5200640 1 Processor -- start 2024-04-25T19:03:08.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.025+0000 7f73a5c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7398033ec0 0x7f73980ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f563c06fa00 con 0x7f563c06f050 2024-04-25T19:03:08.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.025+0000 7f9de5200640 1 -- start start 2024-04-25T19:03:08.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.025+0000 7f73a5c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f739804a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.025+0000 7fe1a5a00640 1 Processor -- start 2024-04-25T19:03:08.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792a00640 1 -- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7940d48b0 msgr2=0x7fc7940d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792a00640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7940d48b0 0x7fc7940d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792a00640 1 -- 172.21.15.123:0/1637566531 >> v1:172.21.15.146:6789/0 conn(0x7fc794160330 legacy=0x7fc7940d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.024+0000 7fc792a00640 1 -- 172.21.15.123:0/1637566531 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79406cd30 con 0x7fc79406c570 2024-04-25T19:03:08.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f73a5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 0x7f739817d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f73a5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7398050870 con 0x7f7398048330 2024-04-25T19:03:08.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f562c01a070 con 0x7f563c06f050 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f73a5c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7398050590 con 0x7f7398033ec0 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f73a5c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73980509f0 con 0x7f7398047de0 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f739f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 0x7f739817d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f739f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 0x7f739817d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34244/0 (socket says 172.21.15.123:34244) 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f739f400640 1 -- 172.21.15.123:0/348547758 learned_addr learned my addr 172.21.15.123:0/348547758 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7fe1a5a00640 1 -- start start 2024-04-25T19:03:08.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.023+0000 7fee18a00640 1 -- 172.21.15.123:0/3029527351 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1406d1c0 con 0x7fee140d0a90 2024-04-25T19:03:08.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7fee13e00640 1 --2- 172.21.15.123:0/3029527351 >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 0x7fee1406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f739e000640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f739804a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f739ea00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7398033ec0 0x7f73980ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f1da8800640 1 -- 172.21.15.123:0/2164041811 shutdown_connections 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc0b2010 0x7f83bc158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc15d070 0x7f83bc15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 0x7f83bc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0d2b90 con 0x7f83bc161400 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0b4000 con 0x7f83bc0b2010 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.026+0000 7f83bbe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc0d2d10 con 0x7f83bc15d070 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f83bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 0x7f83bc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f1da8800640 1 -- 172.21.15.123:0/2164041811 wait complete. 2024-04-25T19:03:08.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fee18a00640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee140d0a90 0x7fee140d0ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee140b5320 tx=0x7fee0c02a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c9873ce275538b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fc792a00640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc79406c570 0x7fc79406daf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc784009d90 tx=0x7fc78402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec8e30c32416858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fc792000640 1 -- 172.21.15.123:0/1637566531 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784007740 con 0x7fc79406c570 2024-04-25T19:03:08.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fc792000640 1 -- 172.21.15.123:0/1637566531 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7840078e0 con 0x7fc79406c570 2024-04-25T19:03:08.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fc792000640 1 -- 172.21.15.123:0/1637566531 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784007c90 con 0x7fc79406c570 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f83baa00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc0b2010 0x7f83bc158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f83b3e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc15d070 0x7f83bc15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f83bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 0x7f83bc1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34256/0 (socket says 172.21.15.123:34256) 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7f83bb400640 1 -- 172.21.15.123:0/1621982593 learned_addr learned my addr 172.21.15.123:0/1621982593 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 -- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 msgr2=0x7f739804a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7398047de0 0x7f739804a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 -- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7398033ec0 msgr2=0x7f73980ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.027+0000 7fee12a00640 1 -- 172.21.15.123:0/3029527351 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c037070 con 0x7fee140d0a90 2024-04-25T19:03:08.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7fee12a00640 1 -- 172.21.15.123:0/3029527351 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c032440 con 0x7fee140d0a90 2024-04-25T19:03:08.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7398033ec0 0x7f73980ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739804acd0 con 0x7f7398048330 2024-04-25T19:03:08.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f9de5200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 0x7f9dd81798f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f9de5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd8058b90 0x7f9dd80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f739f400640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 0x7f739817d940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73a0074ec0 tx=0x7f73a0069ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f83bb400640 1 -- 172.21.15.123:0/1621982593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc15d070 msgr2=0x7f83bc15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a008cc30 con 0x7f7398048330 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0090030 con 0x7f7398048330 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a008cc30 con 0x7f7398048330 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f83bb400640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83bc15d070 0x7f83bc15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f83bb400640 1 -- 172.21.15.123:0/1621982593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc0b2010 msgr2=0x7f83bc158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f83bb400640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83bc0b2010 0x7f83bc158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f5633400640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f561c074530 0x7f561c076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f562c094560 con 0x7f563c06f050 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f5633400640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f561c07dcf0 0x7f561c080180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f562c011e50 con 0x7f561c07dcf0 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.028+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f562c09b050 con 0x7f563c06f050 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f73a5c00640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739804afb0 con 0x7f7398048330 2024-04-25T19:03:08.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f563aa00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f561c074530 0x7f561c076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7fee12a00640 1 -- 172.21.15.123:0/3029527351 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c032850 con 0x7fee140d0a90 2024-04-25T19:03:08.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7fc799800640 1 -- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc79406c570 msgr2=0x7fc79406daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7fc799800640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc79406c570 0x7fc79406daf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc784009d90 tx=0x7fc78402a680 comp rx=0 tx=0).stop 2024-04-25T19:03:08.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7f563be00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f561c07dcf0 0x7f561c080180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 -- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee140d0a90 msgr2=0x7fee140d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee140d0a90 0x7fee140d0ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee140b5320 tx=0x7fee0c02a200 comp rx=0 tx=0).stop 2024-04-25T19:03:08.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.029+0000 7f73a5c00640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739804ce60 con 0x7f7398048330 2024-04-25T19:03:08.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 -- 172.21.15.123:0/3029527351 shutdown_connections 2024-04-25T19:03:08.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7f563aa00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f561c074530 0x7f561c076a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f563c0b5360 tx=0x7f5628009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee140d0a90 0x7fee140d0ea0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 --2- 172.21.15.123:0/3029527351 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee14163d00 0x7fee140b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 --2- 172.21.15.123:0/3029527351 >> v2:172.21.15.123:3300/0 conn(0x7fee1406dca0 0x7fee1406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 -- 172.21.15.123:0/3029527351 >> 172.21.15.123:0/3029527351 conn(0x7fee140d6510 msgr2=0x7fee140d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fc799800640 1 -- 172.21.15.123:0/1637566531 shutdown_connections 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fc799800640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7940d48b0 0x7fc7940d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fc799800640 1 --2- 172.21.15.123:0/1637566531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc79406c570 0x7fc79406daf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fc799800640 1 -- 172.21.15.123:0/1637566531 >> 172.21.15.123:0/1637566531 conn(0x7fc7940d63f0 msgr2=0x7fc7940d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 -- 172.21.15.123:0/3029527351 shutdown_connections 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7f9ddf400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 0x7f9dd81798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7f9ddf400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 0x7f9dd81798f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47498/0 (socket says 172.21.15.123:47498) 2024-04-25T19:03:08.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f9ddf400640 1 -- 172.21.15.123:0/3453937193 learned_addr learned my addr 172.21.15.123:0/3453937193 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7f83bb400640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc15bad0 con 0x7f83bc161400 2024-04-25T19:03:08.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f9de5200640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd80e3a90 0x7f9dd80d0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f83bb400640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 0x7f83bc1558d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009b40 tx=0x7f83b400b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f1da8800640 1 Processor -- start 2024-04-25T19:03:08.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7fc799800640 1 -- 172.21.15.123:0/1637566531 shutdown_connections 2024-04-25T19:03:08.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7fc799800640 1 -- 172.21.15.123:0/1637566531 wait complete. 2024-04-25T19:03:08.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a006da60 0x7fe1a01fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f1da8800640 1 -- start start 2024-04-25T19:03:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.031+0000 7f9ddea00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd8058b90 0x7f9dd80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7374001030 con 0x7f7398048330 2024-04-25T19:03:08.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4003990 con 0x7f83bc161400 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4003b50 con 0x7f83bc161400 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4008990 con 0x7f83bc161400 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a0163ae0 0x7fe1a00cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a00b67d0 0x7fe1a00b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00d51f0 con 0x7fe1a006da60 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00d4f10 con 0x7fe1a00b67d0 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe1a5a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a01605f0 con 0x7fe1a0163ae0 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe19fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a006da60 0x7fe1a01fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f563be00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f561c07dcf0 0x7f561c080180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.030+0000 7fee19400640 1 -- 172.21.15.123:0/3029527351 wait complete. 2024-04-25T19:03:08.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe19fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a006da60 0x7fe1a01fd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34260/0 (socket says 172.21.15.123:34260) 2024-04-25T19:03:08.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7fe19fe00640 1 -- 172.21.15.123:0/3291340063 learned_addr learned my addr 172.21.15.123:0/3291340063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.032+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd803c840 con 0x7f9dd8058b90 2024-04-25T19:03:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a0163ae0 0x7fe1a00cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe1a5000640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a00b67d0 0x7fe1a00b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc15bdb0 con 0x7f83bc161400 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc159100 con 0x7f83bc161400 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8037d70 con 0x7f9dd80e3a90 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd803c9c0 con 0x7f9dd80073d0 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 -- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a00b67d0 msgr2=0x7fe1a00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a00b67d0 0x7fe1a00b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 -- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a006da60 msgr2=0x7fe1a01fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a006da60 0x7fe1a01fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7fe19f400640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a00b7120 con 0x7fe1a0163ae0 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.034+0000 7fe19f400640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a0163ae0 0x7fe1a00cf1e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00e8a0 tx=0x7fe18c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.034+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== osd.2 v2:172.21.15.123:6800/3468692589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f562c011e50 con 0x7f561c07dcf0 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.034+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83bc050740 con 0x7f83bc161400 2024-04-25T19:03:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.033+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73a006d6f0 con 0x7f7398048330 2024-04-25T19:03:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fc799800640 1 Processor -- start 2024-04-25T19:03:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.034+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c017070 con 0x7fe1a0163ae0 2024-04-25T19:03:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a00c9f20 con 0x7fe1a0163ae0 2024-04-25T19:03:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a00ca670 con 0x7fe1a0163ae0 2024-04-25T19:03:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c00c940 con 0x7fe1a0163ae0 2024-04-25T19:03:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c009e60 con 0x7fe1a0163ae0 2024-04-25T19:03:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.034+0000 7f9ddfe00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd80e3a90 0x7f9dd80d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.035+0000 7fc799800640 1 -- start start 2024-04-25T19:03:08.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f1da8800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1da0070850 0x7f1da00b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f1da8800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7fee19400640 1 Processor -- start 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f1da8800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 0x7f1da015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f1da8800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1da00cc570 con 0x7f1da0160070 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f1da8800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0160480 con 0x7f1da015c460 2024-04-25T19:03:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f1da8800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1da00cde60 con 0x7f1da0070850 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f9ddea00640 1 -- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 msgr2=0x7f9dd81798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f9ddea00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 0x7f9dd81798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f9ddea00640 1 -- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd80e3a90 msgr2=0x7f9dd80d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f9ddea00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd80e3a90 0x7f9dd80d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f9ddea00640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd80cf340 con 0x7f9dd8058b90 2024-04-25T19:03:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7fee19400640 1 -- start start 2024-04-25T19:03:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f1da6800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 0x7f1da015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f1da6800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 0x7f1da015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48360/0 (socket says 172.21.15.123:48360) 2024-04-25T19:03:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f9ddfe00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dd80e3a90 0x7f9dd80d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f1da6800640 1 -- 172.21.15.123:0/618743693 learned_addr learned my addr 172.21.15.123:0/618743693 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe178001030 con 0x7fe1a0163ae0 2024-04-25T19:03:08.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f9ddea00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd8058b90 0x7f9dd80d03e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc031000 tx=0x7f9dcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f9ddf400640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dd80073d0 0x7f9dd81798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7fc799800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 0x7fc794161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f7396a00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73800746b0 0x7f7380076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.036+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f73a0111190 con 0x7f7398048330 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f7396a00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f738007de80 0x7f7380080310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73800809e0 con 0x7f738007de80 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73a0118050 con 0x7f7398048330 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.037+0000 7f739ea00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73800746b0 0x7f7380076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f739ea00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73800746b0 0x7f7380076bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7390034000 tx=0x7f739002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc037050 con 0x7f9dd8058b90 2024-04-25T19:03:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc038040 con 0x7f9dd8058b90 2024-04-25T19:03:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83b4008b30 con 0x7f83bc161400 2024-04-25T19:03:08.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7fc799800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940d48b0 0x7fc7940ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7fc799800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc794160330 0x7fc7940cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc799800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940b9310 con 0x7fc79406c570 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc799800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc79406d2f0 con 0x7fc7940d48b0 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc799800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7940b9490 con 0x7fc794160330 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc03c850 con 0x7f9dd8058b90 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f1da6800640 1 -- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1da0070850 msgr2=0x7f1da00b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f1da6800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1da0070850 0x7f1da00b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f1da6800640 1 -- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 msgr2=0x7f1da0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.038+0000 7f1da5400640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7f1da6800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da0160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7f1da6800640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da0160b70 con 0x7f1da015c460 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7f1da5400640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1da0160070 0x7f1da0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee1406dca0 0x7fee1415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 0x7fee1414a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7f739e000640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f738007de80 0x7f7380080310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee14163d00 0x7fee141fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc792a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 0x7fc794161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee140b89f0 con 0x7fee140d0a90 2024-04-25T19:03:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee1406d780 con 0x7fee14163d00 2024-04-25T19:03:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fee19400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee140d46b0 con 0x7fee1406dca0 2024-04-25T19:03:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc792a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 0x7fc794161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34268/0 (socket says 172.21.15.123:34268) 2024-04-25T19:03:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc792a00640 1 -- 172.21.15.123:0/1585757628 learned_addr learned my addr 172.21.15.123:0/1585757628 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc78a000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940d48b0 0x7fc7940ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.039+0000 7fc793400640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc794160330 0x7fc7940cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7f1da6800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 0x7f1da015c870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c700 tx=0x7f1d9c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fee13400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 0x7fee1414a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7f739e000640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f738007de80 0x7f7380080310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd80cf620 con 0x7f9dd8058b90 2024-04-25T19:03:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fee13400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 0x7fee1414a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34294/0 (socket says 172.21.15.123:34294) 2024-04-25T19:03:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fee13400640 1 -- 172.21.15.123:0/1898814826 learned_addr learned my addr 172.21.15.123:0/1898814826 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fc792a00640 1 -- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc794160330 msgr2=0x7fc7940cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fc792a00640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc794160330 0x7fc7940cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7fc792a00640 1 -- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940d48b0 msgr2=0x7fc7940ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd80cfbe0 con 0x7f9dd8058b90 2024-04-25T19:03:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.040+0000 7f83b2a00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8398074580 0x7f8398076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c018070 con 0x7f1da015c460 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00ce10 con 0x7f1da015c460 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00a880 con 0x7f1da015c460 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01615b0 con 0x7f1da015c460 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01612d0 con 0x7f1da015c460 2024-04-25T19:03:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f83b4095e70 con 0x7f83bc161400 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f83b2a00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f839807dd50 0x7f83980801e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83b4017ec0 con 0x7f839807dd50 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83b4096260 con 0x7f83bc161400 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fee18a00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee14163d00 0x7fee141fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fee13e00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee1406dca0 0x7fee1415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fc792a00640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940d48b0 0x7fc7940ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fc792a00640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc794157920 con 0x7fc79406c570 2024-04-25T19:03:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9dd80d0e60 con 0x7f9dd8058b90 2024-04-25T19:03:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7f83baa00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8398074580 0x7f8398076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.041+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe18c01a070 con 0x7fe1a0163ae0 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fe197e00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1800744e0 0x7fe1800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.042+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe18c095740 con 0x7fe1a0163ae0 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fe197e00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7fe18007dca0 0x7fe180080130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe18c016ec0 con 0x7fe18007dca0 2024-04-25T19:03:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe18c095b30 con 0x7fe1a0163ae0 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fc78a000640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940d48b0 0x7fc7940ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fee13400640 1 -- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee1406dca0 msgr2=0x7fee1415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fee13400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee1406dca0 0x7fee1415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fee13400640 1 -- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee14163d00 msgr2=0x7fee141fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fee13400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee14163d00 0x7fee141fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fee13400640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1415c780 con 0x7fee140d0a90 2024-04-25T19:03:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d7c001030 con 0x7f1da015c460 2024-04-25T19:03:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== osd.3 v2:172.21.15.123:6824/3224162034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73800809e0 con 0x7f738007de80 2024-04-25T19:03:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7f83b3e00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f839807dd50 0x7f83980801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fee13400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 0x7fee1414a6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee0000e8a0 tx=0x7fee0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00017070 con 0x7fee140d0a90 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fc792a00640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 0x7fc794161710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc784000f80 tx=0x7fc784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784009450 con 0x7fc79406c570 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc784004240 con 0x7fc79406c570 2024-04-25T19:03:08.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784004610 con 0x7fc79406c570 2024-04-25T19:03:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9dcc045020 con 0x7f9dd8058b90 2024-04-25T19:03:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0000c940 con 0x7fee140d0a90 2024-04-25T19:03:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.044+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0000cd30 con 0x7fee140d0a90 2024-04-25T19:03:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1415bc00 con 0x7fee140d0a90 2024-04-25T19:03:08.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.046+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc794160950 con 0x7fc79406c570 2024-04-25T19:03:08.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1415c1c0 con 0x7fee140d0a90 2024-04-25T19:03:08.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.043+0000 7fe1a5000640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7fe18007dca0 0x7fe180080130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7fe1a5000640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7fe18007dca0 0x7fe180080130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.046+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc794160f10 con 0x7fc79406c570 2024-04-25T19:03:08.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.047+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee14050740 con 0x7fee140d0a90 2024-04-25T19:03:08.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.047+0000 7fe19fe00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1800744e0 0x7fe1800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== osd.1 v2:172.21.15.123:6807/1658520408 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe18c016ec0 con 0x7fe18007dca0 2024-04-25T19:03:08.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc794076a40 con 0x7fc79406c570 2024-04-25T19:03:08.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.046+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1d9c01b070 con 0x7f1da015c460 2024-04-25T19:03:08.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1d9a000640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d840745b0 0x7f1d84076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1d9c094b10 con 0x7f1da015c460 2024-04-25T19:03:08.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7fe19fe00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1800744e0 0x7fe1800769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe190000f60 tx=0x7fe190009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1da5e00640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d840745b0 0x7f1d84076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1d9a000640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7f1d8407dd00 0x7f1d84080190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d84080860 con 0x7f1d8407dd00 2024-04-25T19:03:08.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c05af20 con 0x7f1da015c460 2024-04-25T19:03:08.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7f83b3e00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f839807dd50 0x7f83980801e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.045+0000 7f83baa00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8398074580 0x7f8398076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009560 tx=0x7f83ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.048+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== osd.0 v2:172.21.15.123:6816/1214013409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83b4017ec0 con 0x7f839807dd50 2024-04-25T19:03:08.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.049+0000 7f9dd3400640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9dc0074500 0x7f9dc0076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f1da5400640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7f1d8407dd00 0x7f1d84080190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.049+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9dcc0b5ec0 con 0x7f9dd8058b90 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f9dd3400640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f9dc007dcc0 0x7f9dc0080150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dcc038e50 con 0x7f9dc007dcc0 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc0b63a0 con 0x7f9dd8058b90 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f9ddf400640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9dc0074500 0x7f9dc0076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7f9ddfe00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f9dc007dcc0 0x7f9dc0080150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee0001a070 con 0x7fee140d0a90 2024-04-25T19:03:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.050+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc78404c420 con 0x7fc79406c570 2024-04-25T19:03:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.051+0000 7f1da5e00640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d840745b0 0x7f1d84076ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d94034000 tx=0x7f1d9402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.051+0000 7fc78b400640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc770074500 0x7fc770076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7f9ddfe00640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f9dc007dcc0 0x7f9dc0080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc7840b66f0 con 0x7fc79406c570 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fc78b400640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7fc77007dcc0 0x7fc770080150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc770080820 con 0x7fc77007dcc0 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7840b6bd0 con 0x7fc79406c570 2024-04-25T19:03:08.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7f1da5400640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7f1d8407dd00 0x7f1d84080190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fee0be00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fedf4074530 0x7fedf4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7fc78a000640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc770074500 0x7fc770076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fc793400640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7fc77007dcc0 0x7fc770080150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.052+0000 7f9ddf400640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9dc0074500 0x7f9dc0076a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9de00512d0 tx=0x7f9de0070040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fee00095c60 con 0x7fee140d0a90 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fee0be00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7fedf407dcf0 0x7fedf4080180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedf4080850 con 0x7fedf407dcf0 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee00096010 con 0x7fee140d0a90 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.053+0000 7fee13e00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fedf4074530 0x7fedf4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.054+0000 7fc793400640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7fc77007dcc0 0x7fc770080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.054+0000 7fc78a000640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc770074500 0x7fc770076a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7940c94b0 tx=0x7fc77c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.054+0000 7fee13e00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fedf4074530 0x7fedf4076a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee04003fd0 tx=0x7fee040039f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.054+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== osd.5 v2:172.21.15.146:6800/2028526652 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dcc038e50 con 0x7f9dc007dcc0 2024-04-25T19:03:08.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.056+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== osd.4 v2:172.21.15.146:6801/1889014083 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d84080860 con 0x7f1d8407dd00 2024-04-25T19:03:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.057+0000 7fee18a00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7fedf407dcf0 0x7fedf4080180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.058+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== osd.7 v2:172.21.15.146:6816/3816144943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc770080820 con 0x7fc77007dcc0 2024-04-25T19:03:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.059+0000 7fee18a00640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7fedf407dcf0 0x7fedf4080180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.061+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== osd.6 v2:172.21.15.146:6824/510483223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedf4080850 con 0x7fedf407dcf0 2024-04-25T19:03:08.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.070+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 --> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f563c050740 con 0x7f561c07dcf0 2024-04-25T19:03:08.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.071+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 --> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe178002dd0 con 0x7fe18007dca0 2024-04-25T19:03:08.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.071+0000 7f5633400640 1 -- 172.21.15.123:0/4184441548 <== osd.2 v2:172.21.15.123:6800/3468692589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f563c050740 con 0x7f561c07dcf0 2024-04-25T19:03:08.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.072+0000 7fe197e00640 1 -- 172.21.15.123:0/3291340063 <== osd.1 v2:172.21.15.123:6807/1658520408 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe178002dd0 con 0x7fe18007dca0 2024-04-25T19:03:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.072+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7fe18007dca0 msgr2=0x7fe180080130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.072+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7fe18007dca0 0x7fe180080130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.072+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f561c07dcf0 msgr2=0x7f561c080180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.072+0000 7f5640e00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f561c07dcf0 0x7f561c080180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1800744e0 msgr2=0x7fe1800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1800744e0 0x7fe1800769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe190000f60 tx=0x7fe190009040 comp rx=0 tx=0).stop 2024-04-25T19:03:08.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a0163ae0 msgr2=0x7fe1a00cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 --2- 172.21.15.123:0/3291340063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1a0163ae0 0x7fe1a00cf1e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00e8a0 tx=0x7fe18c00ed60 comp rx=0 tx=0).stop 2024-04-25T19:03:08.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5000640 1 -- 172.21.15.123:0/3291340063 reap_dead start 2024-04-25T19:03:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f561c074530 msgr2=0x7f561c076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f561c074530 0x7f561c076a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f563c0b5360 tx=0x7f5628009190 comp rx=0 tx=0).stop 2024-04-25T19:03:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c06f050 msgr2=0x7f563c09fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 --2- 172.21.15.123:0/4184441548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f563c06f050 0x7f563c09fde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f563c0d0890 tx=0x7f562c00c760 comp rx=0 tx=0).stop 2024-04-25T19:03:08.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 shutdown_connections 2024-04-25T19:03:08.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 >> 172.21.15.123:0/3291340063 conn(0x7fe1a00d6510 msgr2=0x7fe1a00d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f563be00640 1 -- 172.21.15.123:0/4184441548 reap_dead start 2024-04-25T19:03:08.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 shutdown_connections 2024-04-25T19:03:08.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 >> 172.21.15.123:0/4184441548 conn(0x7f563c09b4a0 msgr2=0x7f563c0cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 shutdown_connections 2024-04-25T19:03:08.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.075+0000 7f5640e00640 1 -- 172.21.15.123:0/4184441548 wait complete. 2024-04-25T19:03:08.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.076+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 shutdown_connections 2024-04-25T19:03:08.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.076+0000 7fe1a5a00640 1 -- 172.21.15.123:0/3291340063 wait complete. 2024-04-25T19:03:08.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.082+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 --> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7374002dd0 con 0x7f738007de80 2024-04-25T19:03:08.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.083+0000 7f7396a00640 1 -- 172.21.15.123:0/348547758 <== osd.3 v2:172.21.15.123:6824/3224162034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7374002dd0 con 0x7f738007de80 2024-04-25T19:03:08.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.084+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 --> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9dd80cf0b0 con 0x7f9dc007dcc0 2024-04-25T19:03:08.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.084+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f738007de80 msgr2=0x7f7380080310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.084+0000 7f738be00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f738007de80 0x7f7380080310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.084+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 --> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83bc0cf620 con 0x7f839807dd50 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73800746b0 msgr2=0x7f7380076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f738be00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73800746b0 0x7f7380076bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7390034000 tx=0x7f739002e000 comp rx=0 tx=0).stop 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 msgr2=0x7f739817d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f738be00640 1 --2- 172.21.15.123:0/348547758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398048330 0x7f739817d940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73a0074ec0 tx=0x7f73a0069ba0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f739f400640 1 -- 172.21.15.123:0/348547758 reap_dead start 2024-04-25T19:03:08.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.085+0000 7f83b2a00640 1 -- 172.21.15.123:0/1621982593 <== osd.0 v2:172.21.15.123:6816/1214013409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83bc0cf620 con 0x7f839807dd50 2024-04-25T19:03:08.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f9dd3400640 1 -- 172.21.15.123:0/3453937193 <== osd.5 v2:172.21.15.146:6800/2028526652 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9dd80cf0b0 con 0x7f9dc007dcc0 2024-04-25T19:03:08.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f839807dd50 msgr2=0x7f83980801e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f83bbe00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f839807dd50 0x7f83980801e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f9dc007dcc0 msgr2=0x7f9dc0080150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f9de5200640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f9dc007dcc0 0x7f9dc0080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.086+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8398074580 msgr2=0x7f8398076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bbe00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8398074580 0x7f8398076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009560 tx=0x7f83ac009190 comp rx=0 tx=0).stop 2024-04-25T19:03:08.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 msgr2=0x7f83bc1558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bbe00640 1 --2- 172.21.15.123:0/1621982593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83bc161400 0x7f83bc1558d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009b40 tx=0x7f83b400b4e0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bb400640 1 -- 172.21.15.123:0/1621982593 reap_dead start 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 shutdown_connections 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 >> 172.21.15.123:0/1621982593 conn(0x7f83bc09b4a0 msgr2=0x7f83bc070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9dc0074500 msgr2=0x7f9dc0076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9dc0074500 0x7f9dc0076a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9de00512d0 tx=0x7f9de0070040 comp rx=0 tx=0).stop 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd8058b90 msgr2=0x7f9dd80d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 --2- 172.21.15.123:0/3453937193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd8058b90 0x7f9dd80d03e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc031000 tx=0x7f9dcc0046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 shutdown_connections 2024-04-25T19:03:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 >> 172.21.15.123:0/348547758 conn(0x7f7398054d20 msgr2=0x7f7398053580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9ddfe00640 1 -- 172.21.15.123:0/3453937193 reap_dead start 2024-04-25T19:03:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 shutdown_connections 2024-04-25T19:03:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 shutdown_connections 2024-04-25T19:03:08.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 >> 172.21.15.123:0/3453937193 conn(0x7f9dd803abf0 msgr2=0x7f9dd803dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.087+0000 7f738be00640 1 -- 172.21.15.123:0/348547758 wait complete. 2024-04-25T19:03:08.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 shutdown_connections 2024-04-25T19:03:08.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7f9de5200640 1 -- 172.21.15.123:0/3453937193 wait complete. 2024-04-25T19:03:08.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 shutdown_connections 2024-04-25T19:03:08.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7f83bbe00640 1 -- 172.21.15.123:0/1621982593 wait complete. 2024-04-25T19:03:08.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 --> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee1415b9b0 con 0x7fedf407dcf0 2024-04-25T19:03:08.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.088+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 --> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d7c002dd0 con 0x7f1d8407dd00 2024-04-25T19:03:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.089+0000 7fee0be00640 1 -- 172.21.15.123:0/1898814826 <== osd.6 v2:172.21.15.146:6824/510483223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fee1415b9b0 con 0x7fedf407dcf0 2024-04-25T19:03:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.090+0000 7f1d9a000640 1 -- 172.21.15.123:0/618743693 <== osd.4 v2:172.21.15.146:6801/1889014083 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1d7c002dd0 con 0x7f1d8407dd00 2024-04-25T19:03:08.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7f1d8407dd00 msgr2=0x7f1d84080190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7f1d8407dd00 0x7f1d84080190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d840745b0 msgr2=0x7f1d84076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d840745b0 0x7f1d84076ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d94034000 tx=0x7f1d9402c000 comp rx=0 tx=0).stop 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 msgr2=0x7f1da015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da8800640 1 --2- 172.21.15.123:0/618743693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1da015c460 0x7f1da015c870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c700 tx=0x7f1d9c00cbc0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7f1da6800640 1 -- 172.21.15.123:0/618743693 reap_dead start 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7fedf407dcf0 msgr2=0x7fedf4080180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7fedf407dcf0 0x7fedf4080180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fedf4074530 msgr2=0x7fedf4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fedf4074530 0x7fedf4076a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee04003fd0 tx=0x7fee040039f0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 msgr2=0x7fee1414a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.091+0000 7fee19400640 1 --2- 172.21.15.123:0/1898814826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee140d0a90 0x7fee1414a6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee0000e8a0 tx=0x7fee0000ed60 comp rx=0 tx=0).stop 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.092+0000 7fee18a00640 1 -- 172.21.15.123:0/1898814826 reap_dead start 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.092+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 shutdown_connections 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.092+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 >> 172.21.15.123:0/1898814826 conn(0x7fee140d6510 msgr2=0x7fee1415fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.093+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 shutdown_connections 2024-04-25T19:03:08.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.093+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 >> 172.21.15.123:0/618743693 conn(0x7f1da00ce1b0 msgr2=0x7f1da00b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.093+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 shutdown_connections 2024-04-25T19:03:08.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.093+0000 7f1da8800640 1 -- 172.21.15.123:0/618743693 wait complete. 2024-04-25T19:03:08.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.094+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 shutdown_connections 2024-04-25T19:03:08.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.094+0000 7fee19400640 1 -- 172.21.15.123:0/1898814826 wait complete. 2024-04-25T19:03:08.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.094+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 --> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7940bb800 con 0x7fc77007dcc0 2024-04-25T19:03:08.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.095+0000 7fc78b400640 1 -- 172.21.15.123:0/1585757628 <== osd.7 v2:172.21.15.146:6816/3816144943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7940bb800 con 0x7fc77007dcc0 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.096+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7fc77007dcc0 msgr2=0x7fc770080150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.096+0000 7fc799800640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7fc77007dcc0 0x7fc770080150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.099+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc770074500 msgr2=0x7fc770076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.099+0000 7fc799800640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc770074500 0x7fc770076a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7940c94b0 tx=0x7fc77c009040 comp rx=0 tx=0).stop 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.099+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 msgr2=0x7fc794161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.099+0000 7fc799800640 1 --2- 172.21.15.123:0/1585757628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc79406c570 0x7fc794161710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc784000f80 tx=0x7fc784004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.099+0000 7fc793400640 1 -- 172.21.15.123:0/1585757628 reap_dead start 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.100+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 shutdown_connections 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.100+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 >> 172.21.15.123:0/1585757628 conn(0x7fc7940d63f0 msgr2=0x7fc79415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.100+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 shutdown_connections 2024-04-25T19:03:08.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.100+0000 7fc799800640 1 -- 172.21.15.123:0/1585757628 wait complete. 2024-04-25T19:03:08.129 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:08.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T19:03:08.130 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:08.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T19:03:08.131 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:08.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T19:03:08.132 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:08.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T19:03:08.133 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:08.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T19:03:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:08.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T19:03:08.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T19:03:08.138 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T19:03:08.138 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T19:03:08.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.527+0000 7f2e3e600640 1 Processor -- start 2024-04-25T19:03:08.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.527+0000 7f2e3e600640 1 -- start start 2024-04-25T19:03:08.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.529+0000 7fa2a6e00640 1 Processor -- start 2024-04-25T19:03:08.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.530+0000 7f2e3e600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 0x7f2e30053920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.530+0000 7f2e3e600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2e3003c4e0 0x7f2e30053e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.530+0000 7f2e3e600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e30039880 con 0x7f2e30007800 2024-04-25T19:03:08.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.530+0000 7f2e3e600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2e30038820 con 0x7f2e3003c4e0 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.530+0000 7f2e3e600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e30038ac0 con 0x7f2e30053160 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.531+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 0x7f2e30053920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.531+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 0x7f2e30053920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47548/0 (socket says 172.21.15.123:47548) 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.531+0000 7f2e3d200640 1 -- 172.21.15.123:0/3919636026 learned_addr learned my addr 172.21.15.123:0/3919636026 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.531+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/3919636026 >> v2:172.21.15.123:3300/0 conn(0x7f2e3003c4e0 0x7f2e30053e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.531+0000 7fa2a6e00640 1 -- start start 2024-04-25T19:03:08.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.532+0000 7f2e3dc00640 1 -- 172.21.15.123:0/3919636026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 msgr2=0x7f2e30053920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.532+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/3919636026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 0x7f2e30053920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.532+0000 7f2e3dc00640 1 -- 172.21.15.123:0/3919636026 >> v1:172.21.15.123:6789/0 conn(0x7f2e30007800 legacy=0x7f2e30006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.533+0000 7f2e3dc00640 1 -- 172.21.15.123:0/3919636026 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30038da0 con 0x7f2e3003c4e0 2024-04-25T19:03:08.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7fa2a6e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/3919636026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e30053e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e380514f0 tx=0x7f2e380a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c59b81c1b262c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7fa2a6e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2a006c570 0x7fa2a00b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7f2e37400640 1 -- 172.21.15.123:0/3919636026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b4070 con 0x7f2e3003c4e0 2024-04-25T19:03:08.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7fa29fe00640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa2a00d4580 0x7fa2a00d0950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:47488/0 (socket says 172.21.15.123:47488) 2024-04-25T19:03:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7f2e37400640 1 -- 172.21.15.123:0/3919636026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e380a7b30 con 0x7f2e3003c4e0 2024-04-25T19:03:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.535+0000 7f2e37400640 1 -- 172.21.15.123:0/3919636026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380af440 con 0x7f2e3003c4e0 2024-04-25T19:03:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7fa29f400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7fa29f400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47556/0 (socket says 172.21.15.123:47556) 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7fa29fe00640 1 -- 172.21.15.123:0/3752655578 learned_addr learned my addr 172.21.15.123:0/3752655578 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a006cb10 con 0x7fa2a00d4580 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7fa2a4e00640 1 --2- 172.21.15.123:0/3752655578 >> v2:172.21.15.146:3300/0 conn(0x7fa2a006c570 0x7fa2a00b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 -- 172.21.15.123:0/3919636026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 msgr2=0x7f2e30053e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 --2- 172.21.15.123:0/3919636026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e30053e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e380514f0 tx=0x7f2e380a6f90 comp rx=0 tx=0).stop 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.534+0000 7f2866a00640 1 Processor -- start 2024-04-25T19:03:08.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.534+0000 7f2866a00640 1 -- start start 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7f2866a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28600d0be0 0x7f286015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7f2866a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 0x7f28600d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7f2866a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286006d2f0 con 0x7f2860164b50 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a006cdf0 con 0x7fa2a006c570 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a006d0d0 con 0x7fa2a00d1060 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7f2866a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f286006d5d0 con 0x7f2860165670 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.536+0000 7f2866a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f286006d8b0 con 0x7f28600d0be0 2024-04-25T19:03:08.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f285fe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28600d0be0 0x7f286015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 -- 172.21.15.123:0/3919636026 shutdown_connections 2024-04-25T19:03:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 --2- 172.21.15.123:0/3919636026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e30053e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 --2- 172.21.15.123:0/3919636026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30053160 0x7f2e30053920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f2e3e600640 1 -- 172.21.15.123:0/3919636026 >> 172.21.15.123:0/3919636026 conn(0x7f2e30020270 msgr2=0x7f2e3003bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7fa29ea00640 1 -- 172.21.15.123:0/3752655578 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694896964 0 0) 0x7fa2a006cb10 con 0x7fa2a00d4580 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f285fe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28600d0be0 0x7f286015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48388/0 (socket says 172.21.15.123:48388) 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.537+0000 7f285fe00640 1 -- 172.21.15.123:0/1149885246 learned_addr learned my addr 172.21.15.123:0/1149885246 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f2864a00640 1 --2- 172.21.15.123:0/1149885246 >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 0x7f28600d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f285fe00640 1 -- 172.21.15.123:0/1149885246 >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 msgr2=0x7f28600d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f285fe00640 1 --2- 172.21.15.123:0/1149885246 >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 0x7f28600d3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f285fe00640 1 -- 172.21.15.123:0/1149885246 >> v1:172.21.15.146:6789/0 conn(0x7f2860164b50 legacy=0x7f2860164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f285fe00640 1 -- 172.21.15.123:0/1149885246 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286006db90 con 0x7f28600d0be0 2024-04-25T19:03:08.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7fa29ea00640 1 -- 172.21.15.123:0/3752655578 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa284003520 con 0x7fa2a00d4580 2024-04-25T19:03:08.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f2e3e600640 1 -- 172.21.15.123:0/3919636026 shutdown_connections 2024-04-25T19:03:08.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.538+0000 7f2e3e600640 1 -- 172.21.15.123:0/3919636026 wait complete. 2024-04-25T19:03:08.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.539+0000 7f2864a00640 1 --2- 172.21.15.123:0/1149885246 >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 0x7f28600d3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:08.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.539+0000 7fa29f400640 1 -- 172.21.15.123:0/3752655578 >> v2:172.21.15.146:3300/0 conn(0x7fa2a006c570 msgr2=0x7fa2a00b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.539+0000 7fa29f400640 1 --2- 172.21.15.123:0/3752655578 >> v2:172.21.15.146:3300/0 conn(0x7fa2a006c570 0x7fa2a00b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.539+0000 7fa29f400640 1 -- 172.21.15.123:0/3752655578 >> v1:172.21.15.146:6789/0 conn(0x7fa2a00d4580 legacy=0x7fa2a00d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.539+0000 7fa29f400640 1 -- 172.21.15.123:0/3752655578 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a006d3b0 con 0x7fa2a00d1060 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.540+0000 7f285fe00640 1 --2- 172.21.15.123:0/1149885246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28600d0be0 0x7f286015eb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28600b5090 tx=0x7f284c02a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e49ace174c4c4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.541+0000 7f285ea00640 1 -- 172.21.15.123:0/1149885246 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c037070 con 0x7f28600d0be0 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.541+0000 7f2e3e600640 1 Processor -- start 2024-04-25T19:03:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.541+0000 7fa29f400640 1 --2- 172.21.15.123:0/3752655578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097a0 tx=0x7fa29002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d49110d1ad99891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.542+0000 7f285ea00640 1 -- 172.21.15.123:0/1149885246 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c032440 con 0x7f28600d0be0 2024-04-25T19:03:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.542+0000 7f285ea00640 1 -- 172.21.15.123:0/1149885246 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c032810 con 0x7f28600d0be0 2024-04-25T19:03:08.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.542+0000 7f2e3e600640 1 -- start start 2024-04-25T19:03:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.542+0000 7fa29ea00640 1 -- 172.21.15.123:0/3752655578 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290037070 con 0x7fa2a00d1060 2024-04-25T19:03:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.543+0000 7fa29ea00640 1 -- 172.21.15.123:0/3752655578 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290032440 con 0x7fa2a00d1060 2024-04-25T19:03:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.543+0000 7f2866a00640 1 -- 172.21.15.123:0/1149885246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28600d0be0 msgr2=0x7f286015eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.543+0000 7f2866a00640 1 --2- 172.21.15.123:0/1149885246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28600d0be0 0x7f286015eb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28600b5090 tx=0x7f284c02a280 comp rx=0 tx=0).stop 2024-04-25T19:03:08.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.543+0000 7fa29ea00640 1 -- 172.21.15.123:0/3752655578 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290032810 con 0x7fa2a00d1060 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f2866a00640 1 -- 172.21.15.123:0/1149885246 shutdown_connections 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f2866a00640 1 --2- 172.21.15.123:0/1149885246 >> v2:172.21.15.123:3300/0 conn(0x7f2860165670 0x7f28600d3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f2866a00640 1 --2- 172.21.15.123:0/1149885246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28600d0be0 0x7f286015eb60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f2866a00640 1 -- 172.21.15.123:0/1149885246 >> 172.21.15.123:0/1149885246 conn(0x7f28600d6470 msgr2=0x7f28600d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f7cb7800640 1 Processor -- start 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fab6e200640 1 Processor -- start 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 msgr2=0x7fa2a00b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 --2- 172.21.15.123:0/3752655578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097a0 tx=0x7fa29002a180 comp rx=0 tx=0).stop 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f2866a00640 1 -- 172.21.15.123:0/1149885246 shutdown_connections 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 shutdown_connections 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 --2- 172.21.15.123:0/3752655578 >> v2:172.21.15.146:3300/0 conn(0x7fa2a006c570 0x7fa2a00b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 --2- 172.21.15.123:0/3752655578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a00d1060 0x7fa2a00b4df0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 >> 172.21.15.123:0/3752655578 conn(0x7fa2a00d6470 msgr2=0x7fa2a00d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7f7cb7800640 1 -- start start 2024-04-25T19:03:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.545+0000 7f2866a00640 1 -- 172.21.15.123:0/1149885246 wait complete. 2024-04-25T19:03:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.545+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 shutdown_connections 2024-04-25T19:03:08.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.545+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3752655578 wait complete. 2024-04-25T19:03:08.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.546+0000 7f8679000640 1 Processor -- start 2024-04-25T19:03:08.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.544+0000 7fab6e200640 1 -- start start 2024-04-25T19:03:08.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.547+0000 7f2e3e600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30007800 0x7f2e300d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.547+0000 7f2e3e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e3017d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.546+0000 7f8679000640 1 -- start start 2024-04-25T19:03:08.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f7cb7800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cb0165550 0x7f7cb0167970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f7cb7800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb015ce30 con 0x7f7cb00b1990 2024-04-25T19:03:08.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f7cb7800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb015d110 con 0x7f7cb015c670 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e3e600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 0x7f2e300d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e3e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30052570 con 0x7f2e3003c4e0 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e3e600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30039360 con 0x7f2e3017d5a0 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e3e600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300e5350 con 0x7f2e30007800 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e3017d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e3d200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30007800 0x7f2e300d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e3017d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34330/0 (socket says 172.21.15.123:34330) 2024-04-25T19:03:08.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f2e37e00640 1 -- 172.21.15.123:0/623990778 learned_addr learned my addr 172.21.15.123:0/623990778 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.548+0000 7f7cb5800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cb0165550 0x7f7cb0167970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f7cb5800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cb0165550 0x7f7cb0167970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34342/0 (socket says 172.21.15.123:34342) 2024-04-25T19:03:08.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f2866a00640 1 Processor -- start 2024-04-25T19:03:08.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f7cb5800640 1 -- 172.21.15.123:0/2478250289 learned_addr learned my addr 172.21.15.123:0/2478250289 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f2866a00640 1 -- start start 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fab6e200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fab680b0960 0x7fab6816f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fab6e200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab680704f0 con 0x7fab680cd810 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fab6e200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab681649c0 con 0x7fab680b56b0 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fab6e200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fab680b04d0 con 0x7fab680b0960 2024-04-25T19:03:08.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fa2a6e00640 1 Processor -- start 2024-04-25T19:03:08.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb015dbf0 con 0x7f7cb0165550 2024-04-25T19:03:08.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 0x7f2e300d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7f7caf400640 1 -- 172.21.15.123:0/2478250289 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13507912 0 0) 0x7f7cb015ce30 con 0x7f7cb00b1990 2024-04-25T19:03:08.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.549+0000 7fab67400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fab680cd810 0x7fab6806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45302/0 (socket says 172.21.15.123:45302) 2024-04-25T19:03:08.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7fab67400640 1 -- 172.21.15.123:0/3371383720 learned_addr learned my addr 172.21.15.123:0/3371383720 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f8679000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 0x7f86740d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f7caf400640 1 -- 172.21.15.123:0/2478250289 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c94004630 con 0x7f7cb00b1990 2024-04-25T19:03:08.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7fa2a6e00640 1 -- start start 2024-04-25T19:03:08.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f8679000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8674160070 0x7f86740b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f8679000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8674160480 con 0x7f86740b9410 2024-04-25T19:03:08.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f8679000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f867406c710 con 0x7f8674160070 2024-04-25T19:03:08.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f8679000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f867406c9f0 con 0x7f86740b4b30 2024-04-25T19:03:08.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f7cb5800640 1 -- 172.21.15.123:0/2478250289 >> v1:172.21.15.146:6789/0 conn(0x7f7cb015c670 legacy=0x7f7cb00b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f7cb5800640 1 -- 172.21.15.123:0/2478250289 >> v1:172.21.15.123:6789/0 conn(0x7f7cb00b1990 legacy=0x7f7cb00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f7cb5800640 1 -- 172.21.15.123:0/2478250289 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb015d3f0 con 0x7f7cb0165550 2024-04-25T19:03:08.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096289552 0 0) 0x7fab681649c0 con 0x7fab680b56b0 2024-04-25T19:03:08.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab4c003520 con 0x7fab680b56b0 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f2e3dc00640 1 -- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30007800 msgr2=0x7f2e300d6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30007800 0x7f2e300d6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f2e3dc00640 1 -- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 msgr2=0x7f2e3017d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e3017d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.550+0000 7f2e3dc00640 1 -- 172.21.15.123:0/623990778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e300e47e0 con 0x7f2e3017d5a0 2024-04-25T19:03:08.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7fdf7f200640 1 Processor -- start 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 0x7f2860073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 0x7f2860070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f286006d070 con 0x7f28600d0be0 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f286006cef0 con 0x7f2860165670 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f2866a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28600d4500 con 0x7f2860164b50 2024-04-25T19:03:08.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7fdf7f200640 1 -- start start 2024-04-25T19:03:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f8673400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 0x7f86740d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.551+0000 7f8672a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8674160070 0x7f86740b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f8673e00640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f86740b9410 0x7f86740d0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:47530/0 (socket says 172.21.15.123:47530) 2024-04-25T19:03:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f8673e00640 1 -- 172.21.15.123:0/395382870 learned_addr learned my addr 172.21.15.123:0/395382870 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f0af6000640 1 Processor -- start 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 589401552 0 0) 0x7fab4c003520 con 0x7fab680b56b0 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 >> v2:172.21.15.123:3300/0 conn(0x7fab680b0960 msgr2=0x7fab6816f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 --2- 172.21.15.123:0/3371383720 >> v2:172.21.15.123:3300/0 conn(0x7fab680b0960 0x7fab6816f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 >> v1:172.21.15.123:6789/0 conn(0x7fab680cd810 legacy=0x7fab6806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab680707d0 con 0x7fab680b56b0 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fab600019f0 con 0x7fab680b56b0 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f285fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f285fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34352/0 (socket says 172.21.15.123:34352) 2024-04-25T19:03:08.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f285fe00640 1 -- 172.21.15.123:0/553005192 learned_addr learned my addr 172.21.15.123:0/553005192 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 >> v1:172.21.15.146:6789/0 conn(0x7fab680b56b0 legacy=0x7fab6806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab4c003a00 0x7fab4c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c004800 0x7fab4c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 0x7fab4c00ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab680b04d0 con 0x7fab4c004800 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00c330 con 0x7fab4c003a00 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab4c00c530 con 0x7fab4c0053a0 2024-04-25T19:03:08.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f7cb5800640 1 --2- 172.21.15.123:0/2478250289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0167970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009870 tx=0x7f7ca002c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f65d870d1479b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7fab66a00640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c004800 0x7fab4c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.552+0000 7f2e3dc00640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 0x7f2e300d7ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e3806b610 tx=0x7f2e38076180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f0af6000640 1 -- start start 2024-04-25T19:03:08.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f285f400640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 0x7f2860073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f8673400640 1 -- 172.21.15.123:0/395382870 >> v2:172.21.15.146:3300/0 conn(0x7f8674160070 msgr2=0x7f86740b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f8673400640 1 --2- 172.21.15.123:0/395382870 >> v2:172.21.15.146:3300/0 conn(0x7f8674160070 0x7f86740b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f7caf400640 1 -- 172.21.15.123:0/2478250289 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0038070 con 0x7f7cb0165550 2024-04-25T19:03:08.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f7caf400640 1 -- 172.21.15.123:0/2478250289 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca0031710 con 0x7f7cb0165550 2024-04-25T19:03:08.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f7caf400640 1 -- 172.21.15.123:0/2478250289 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0031ae0 con 0x7f7cb0165550 2024-04-25T19:03:08.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f2864a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 0x7f2860070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f285fe00640 1 -- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 msgr2=0x7f2860073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7fab67e00640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 0x7fab4c00ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f285fe00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 0x7f2860073300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b4050 con 0x7f2e3017d5a0 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3017d9b0 con 0x7f2e3017d5a0 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e300e3f40 con 0x7f2e3017d5a0 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f285fe00640 1 -- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 msgr2=0x7f2860070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e380b5040 con 0x7f2e3017d5a0 2024-04-25T19:03:08.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b4050 con 0x7f2e3017d5a0 2024-04-25T19:03:08.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f285fe00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 0x7f2860070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 msgr2=0x7f7cb0167970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f7cb7800640 1 --2- 172.21.15.123:0/2478250289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0167970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009870 tx=0x7f7ca002c530 comp rx=0 tx=0).stop 2024-04-25T19:03:08.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7fa2a6e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a006c570 0x7fa2a0073d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7fa2a6e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a00d4580 0x7fa2a01fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f285fe00640 1 -- 172.21.15.123:0/553005192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2860160680 con 0x7f28600d0be0 2024-04-25T19:03:08.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f285f400640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 0x7f2860073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f2864a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 0x7f2860070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7fa2a6e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 shutdown_connections 2024-04-25T19:03:08.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7f7cb7800640 1 --2- 172.21.15.123:0/2478250289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0167970 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 >> 172.21.15.123:0/2478250289 conn(0x7f7cb0073ce0 msgr2=0x7f7cb009b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f8673400640 1 -- 172.21.15.123:0/395382870 >> v1:172.21.15.146:6789/0 conn(0x7f86740b9410 legacy=0x7f86740d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.553+0000 7f8673400640 1 -- 172.21.15.123:0/395382870 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867406ccd0 con 0x7f86740b4b30 2024-04-25T19:03:08.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.554+0000 7fdf7f200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf8013fbf0 0x7fdf8006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fa2a6e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00d1670 con 0x7fa2a0071b40 2024-04-25T19:03:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fa2a6e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00d14f0 con 0x7fa2a00d4580 2024-04-25T19:03:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fa2a6e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a006dbb0 con 0x7fa2a006c570 2024-04-25T19:03:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fa2a4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 0x7fdf800b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7f8673400640 1 --2- 172.21.15.123:0/395382870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 0x7f86740d4d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f86640097a0 tx=0x7f866402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f331d8d0ce5f6b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8006ded0 con 0x7fdf8006dac0 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7f200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdf80143050 con 0x7fdf8013fbf0 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf800b5c70 con 0x7fdf80142c40 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7d400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdf8006dac0 0x7fdf80142530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45304/0 (socket says 172.21.15.123:45304) 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7d400640 1 -- 172.21.15.123:0/1730418826 learned_addr learned my addr 172.21.15.123:0/1730418826 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fdf7e800640 1 --2- 172.21.15.123:0/1730418826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 0x7fdf800b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7f285fe00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f284c009960 tx=0x7f284c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c041020 con 0x7f28600d0be0 2024-04-25T19:03:08.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f8672000640 1 -- 172.21.15.123:0/395382870 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664037070 con 0x7f86740b4b30 2024-04-25T19:03:08.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 shutdown_connections 2024-04-25T19:03:08.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fa2a4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34370/0 (socket says 172.21.15.123:34370) 2024-04-25T19:03:08.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fa2a4e00640 1 -- 172.21.15.123:0/3892188251 learned_addr learned my addr 172.21.15.123:0/3892188251 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f7cb7800640 1 -- 172.21.15.123:0/2478250289 wait complete. 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f8672000640 1 -- 172.21.15.123:0/395382870 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866402ae40 con 0x7f86740b4b30 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f8672000640 1 -- 172.21.15.123:0/395382870 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664032660 con 0x7f86740b4b30 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c030800 con 0x7f28600d0be0 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c037040 con 0x7f28600d0be0 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2860072540 con 0x7f28600d0be0 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fab67e00640 1 -- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab4c003a00 msgr2=0x7fab4c003e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fab67e00640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab4c003a00 0x7fab4c003e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fab67e00640 1 -- 172.21.15.123:0/3371383720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c004800 msgr2=0x7fab4c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fab67e00640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c004800 0x7fab4c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.555+0000 7fab67e00640 1 -- 172.21.15.123:0/3371383720 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c003520 con 0x7fab4c0053a0 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fab67e00640 1 -- 172.21.15.123:0/3371383720 reap_dead start 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2860072b00 con 0x7f28600d0be0 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fdf7e800640 1 -- 172.21.15.123:0/1730418826 >> v2:172.21.15.146:3300/0 conn(0x7fdf8013fbf0 msgr2=0x7fdf8006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fdf7e800640 1 --2- 172.21.15.123:0/1730418826 >> v2:172.21.15.146:3300/0 conn(0x7fdf8013fbf0 0x7fdf8006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fdf7e800640 1 -- 172.21.15.123:0/1730418826 >> v1:172.21.15.123:6789/0 conn(0x7fdf8006dac0 legacy=0x7fdf80142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7fdf7e800640 1 -- 172.21.15.123:0/1730418826 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf800b5f50 con 0x7fdf80142c40 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.556+0000 7f0af6000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0af01601d0 0x7f0af00d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f0af6000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 0x7f0af006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f0af6000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af006c570 con 0x7f0af006d830 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f0af6000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0af01605e0 con 0x7f0af01601d0 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f0af6000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00d5980 con 0x7f0af00b9070 2024-04-25T19:03:08.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f8679000640 1 -- 172.21.15.123:0/395382870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 msgr2=0x7f86740d4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7f0aef400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0af01601d0 0x7f0af00d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7f8679000640 1 --2- 172.21.15.123:0/395382870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 0x7f86740d4d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f86640097a0 tx=0x7f866402a0f0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7f0aeea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 0x7f0af006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7f0aefe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0af006d830 0x7f0af00d0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45310/0 (socket says 172.21.15.123:45310) 2024-04-25T19:03:08.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7f0aefe00640 1 -- 172.21.15.123:0/116169209 learned_addr learned my addr 172.21.15.123:0/116169209 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7fa2a4e00640 1 -- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a006c570 msgr2=0x7fa2a0073d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7fa2a4e00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a006c570 0x7fa2a0073d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7fa2a4e00640 1 -- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a00d4580 msgr2=0x7fa2a01fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7fa2a4e00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a00d4580 0x7fa2a01fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.557+0000 7fa2a4e00640 1 -- 172.21.15.123:0/3892188251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00722e0 con 0x7fa2a0071b40 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7fab67e00640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 0x7fab4c00ba50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fab680b0090 tx=0x7fab4c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56610f60a2ee3298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.558+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286015e790 con 0x7f28600d0be0 2024-04-25T19:03:08.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f0aeea00640 1 -- 172.21.15.123:0/116169209 >> v2:172.21.15.146:3300/0 conn(0x7f0af01601d0 msgr2=0x7f0af00d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:08.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f0aeea00640 1 --2- 172.21.15.123:0/116169209 >> v2:172.21.15.146:3300/0 conn(0x7f0af01601d0 0x7f0af00d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f0aeea00640 1 -- 172.21.15.123:0/116169209 >> v1:172.21.15.123:6789/0 conn(0x7f0af006d830 legacy=0x7f0af00d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fa2a4e00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa29800efc0 tx=0x7fa29800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f0aeea00640 1 -- 172.21.15.123:0/116169209 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af006c810 con 0x7f0af00b9070 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fdf7e800640 1 --2- 172.21.15.123:0/1730418826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 0x7fdf800b4640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7802a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100e5bb4fd147371 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 -- 172.21.15.123:0/395382870 shutdown_connections 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 --2- 172.21.15.123:0/395382870 >> v2:172.21.15.146:3300/0 conn(0x7f8674160070 0x7f86740b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 --2- 172.21.15.123:0/395382870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b4b30 0x7f86740d4d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 -- 172.21.15.123:0/395382870 >> 172.21.15.123:0/395382870 conn(0x7f86740d65d0 msgr2=0x7f86740b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f7cb7800640 1 Processor -- start 2024-04-25T19:03:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 -- 172.21.15.123:0/395382870 shutdown_connections 2024-04-25T19:03:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298018020 con 0x7fa2a0071b40 2024-04-25T19:03:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e14005290 con 0x7f2e3017d5a0 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fdf7ca00640 1 -- 172.21.15.123:0/1730418826 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007740 con 0x7fdf80142c40 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fdf7ca00640 1 -- 172.21.15.123:0/1730418826 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780078e0 con 0x7fdf80142c40 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58009460 con 0x7fab4c0053a0 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58004240 con 0x7fab4c0053a0 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7fab66000640 1 -- 172.21.15.123:0/3371383720 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5802a850 con 0x7fab4c0053a0 2024-04-25T19:03:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.560+0000 7fdf7ca00640 1 -- 172.21.15.123:0/1730418826 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007cb0 con 0x7fdf80142c40 2024-04-25T19:03:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.560+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3892188251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a00725c0 con 0x7fa2a0071b40 2024-04-25T19:03:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.560+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3892188251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a0072b80 con 0x7fa2a0071b40 2024-04-25T19:03:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7f7cb7800640 1 -- start start 2024-04-25T19:03:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fab6e200640 1 -- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 msgr2=0x7fab4c00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fab6e200640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 0x7fab4c00ba50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fab680b0090 tx=0x7fab4c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fab6e200640 1 -- 172.21.15.123:0/3371383720 shutdown_connections 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fab6e200640 1 --2- 172.21.15.123:0/3371383720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab4c0053a0 0x7fab4c00ba50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fab6e200640 1 -- 172.21.15.123:0/3371383720 >> 172.21.15.123:0/3371383720 conn(0x7fab680d1ae0 msgr2=0x7fab680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 -- 172.21.15.123:0/1730418826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 msgr2=0x7fdf800b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 --2- 172.21.15.123:0/1730418826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 0x7fdf800b4640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7802a550 comp rx=0 tx=0).stop 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 -- 172.21.15.123:0/1730418826 shutdown_connections 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 --2- 172.21.15.123:0/1730418826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf80142c40 0x7fdf800b4640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 --2- 172.21.15.123:0/1730418826 >> v2:172.21.15.146:3300/0 conn(0x7fdf8013fbf0 0x7fdf8006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7fdf7f200640 1 -- 172.21.15.123:0/1730418826 >> 172.21.15.123:0/1730418826 conn(0x7fdf80073200 msgr2=0x7fdf80073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.559+0000 7f8679000640 1 -- 172.21.15.123:0/395382870 wait complete. 2024-04-25T19:03:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2e380c2020 con 0x7f2e3017d5a0 2024-04-25T19:03:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7f8679000640 1 Processor -- start 2024-04-25T19:03:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.560+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa298014480 con 0x7fa2a0071b40 2024-04-25T19:03:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298009e60 con 0x7fa2a0071b40 2024-04-25T19:03:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7f8679000640 1 -- start start 2024-04-25T19:03:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7fdf7f200640 1 -- 172.21.15.123:0/1730418826 shutdown_connections 2024-04-25T19:03:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7fdf7f200640 1 -- 172.21.15.123:0/1730418826 wait complete. 2024-04-25T19:03:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.562+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3892188251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa278005290 con 0x7fa2a0071b40 2024-04-25T19:03:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.561+0000 7f0aeea00640 1 --2- 172.21.15.123:0/116169209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 0x7f0af006d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641d0bfccd895b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.563+0000 7fab6e200640 1 -- 172.21.15.123:0/3371383720 shutdown_connections 2024-04-25T19:03:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.563+0000 7fab6e200640 1 -- 172.21.15.123:0/3371383720 wait complete. 2024-04-25T19:03:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.563+0000 7f2e34c00640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 0x7f2e18076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.563+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e38131cd0 con 0x7f2e3017d5a0 2024-04-25T19:03:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.564+0000 7f2e3d200640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 0x7f2e18076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.564+0000 7f0aee000640 1 -- 172.21.15.123:0/116169209 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8037070 con 0x7f0af00b9070 2024-04-25T19:03:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.564+0000 7f0aee000640 1 -- 172.21.15.123:0/116169209 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad8032440 con 0x7f0af00b9070 2024-04-25T19:03:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.564+0000 7f0aee000640 1 -- 172.21.15.123:0/116169209 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad802ae60 con 0x7f0af00b9070 2024-04-25T19:03:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7f2e3d200640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 0x7f2e18076bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005f80 tx=0x7f2e2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7fdf7f200640 1 Processor -- start 2024-04-25T19:03:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7f8679000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7f0af6000640 1 -- 172.21.15.123:0/116169209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 msgr2=0x7f0af006d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7f0af6000640 1 --2- 172.21.15.123:0/116169209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 0x7f0af006d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802a110 comp rx=0 tx=0).stop 2024-04-25T19:03:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7f8679000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b9410 0x7f86740cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8679000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8674160070 0x7f86740cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 -- 172.21.15.123:0/116169209 shutdown_connections 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 --2- 172.21.15.123:0/116169209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af00b9070 0x7f0af006d2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 --2- 172.21.15.123:0/116169209 >> v2:172.21.15.146:3300/0 conn(0x7f0af01601d0 0x7f0af00d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 -- 172.21.15.123:0/116169209 >> 172.21.15.123:0/116169209 conn(0x7f0af00d5d90 msgr2=0x7f0af00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.565+0000 7fdf7f200640 1 -- start start 2024-04-25T19:03:08.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8679000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86740d0e60 con 0x7f86740b4b30 2024-04-25T19:03:08.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 -- 172.21.15.123:0/116169209 shutdown_connections 2024-04-25T19:03:08.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8679000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86740d1410 con 0x7f8674160070 2024-04-25T19:03:08.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8679000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86740d0fe0 con 0x7f86740b9410 2024-04-25T19:03:08.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8673400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 0x7f7cb01fb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb015c670 0x7f7cb009f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb015ca80 con 0x7f7cb0165550 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb015d760 con 0x7f7cb00b1990 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f7cb7800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb01fbd50 con 0x7f7cb015c670 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8673400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34378/0 (socket says 172.21.15.123:34378) 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8673e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8674160070 0x7f86740cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f8673400640 1 -- 172.21.15.123:0/3663947815 learned_addr learned my addr 172.21.15.123:0/3663947815 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7fab6e200640 1 Processor -- start 2024-04-25T19:03:08.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7fab6e200640 1 -- start start 2024-04-25T19:03:08.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.564+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f284c044020 con 0x7f28600d0be0 2024-04-25T19:03:08.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f2857e00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 0x7f28400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f284c0b62c0 con 0x7f28600d0be0 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cb4e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 0x7f7cb01fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cb4e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 0x7f7cb01fb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48454/0 (socket says 172.21.15.123:48454) 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cb4e00640 1 -- 172.21.15.123:0/1119493321 learned_addr learned my addr 172.21.15.123:0/1119493321 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cb5800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cafe00640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb015c670 0x7f7cb009f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f285f400640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 0x7f28400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f8672a00640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b9410 0x7f86740cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.567+0000 7f7cb5800640 1 -- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb015c670 msgr2=0x7f7cb009f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f7cb5800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb015c670 0x7f7cb009f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f7cb5800640 1 -- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 msgr2=0x7f7cb01fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f7cb5800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 0x7f7cb01fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f7cb5800640 1 -- 172.21.15.123:0/1119493321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb009e2c0 con 0x7f7cb0165550 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f7cb5800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0152f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0033000 tx=0x7f7ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f8673400640 1 -- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b9410 msgr2=0x7f86740cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f285f400640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 0x7f28400769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f286015ec30 tx=0x7f2850009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f8673400640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b9410 0x7f86740cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f8673400640 1 -- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8674160070 msgr2=0x7f86740cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f8673400640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8674160070 0x7f86740cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.566+0000 7f0af6000640 1 -- 172.21.15.123:0/116169209 wait complete. 2024-04-25T19:03:08.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7f8673400640 1 -- 172.21.15.123:0/3663947815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740ca990 con 0x7f86740b4b30 2024-04-25T19:03:08.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.568+0000 7fab6e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab6e200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 0x7fab68166960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab6e200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab6809b8b0 0x7fab68166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab6e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab680b0160 con 0x7fab4c0053a0 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab6e200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab68070d70 con 0x7fab6809b4a0 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab6e200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab68071050 con 0x7fab6809b8b0 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8013fbf0 0x7fdf8013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80142c40 0x7fdf801fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf800b6a30 con 0x7fdf8006dac0 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf800b6510 con 0x7fdf80142c40 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf800b4b80 con 0x7fdf8013fbf0 2024-04-25T19:03:08.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7f8673400640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f866402a6e0 tx=0x7f86640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab67400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34400/0 (socket says 172.21.15.123:34400) 2024-04-25T19:03:08.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab67400640 1 -- 172.21.15.123:0/4233266656 learned_addr learned my addr 172.21.15.123:0/4233266656 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34408/0 (socket says 172.21.15.123:34408) 2024-04-25T19:03:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7de00640 1 -- 172.21.15.123:0/3226638386 learned_addr learned my addr 172.21.15.123:0/3226638386 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab66a00640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 0x7fab68166960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7d400640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8013fbf0 0x7fdf8013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fdf7e800640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80142c40 0x7fdf801fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.569+0000 7fab67e00640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab6809b8b0 0x7fab68166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 -- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8013fbf0 msgr2=0x7fdf8013c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8013fbf0 0x7fdf8013c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 -- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80142c40 msgr2=0x7fdf801fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80142c40 0x7fdf801fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 -- 172.21.15.123:0/3226638386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf801fe6a0 con 0x7fdf8006dac0 2024-04-25T19:03:08.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7f0af6000640 1 Processor -- start 2024-04-25T19:03:08.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab67400640 1 -- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab6809b8b0 msgr2=0x7fab68166ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fdf7de00640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b440 tx=0x7fdf7000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab67400640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab6809b8b0 0x7fab68166ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab67400640 1 -- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 msgr2=0x7fab68166960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab67400640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 0x7fab68166960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab67400640 1 -- 172.21.15.123:0/4233266656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6809d790 con 0x7fab4c0053a0 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7fab66a00640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 0x7fab68166960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:08.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.570+0000 7f0af6000640 1 -- start start 2024-04-25T19:03:08.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.571+0000 7fab67400640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c410 tx=0x7fab5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.572+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f284c04b030 con 0x7f28600d0be0 2024-04-25T19:03:08.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0038040 con 0x7f7cb0165550 2024-04-25T19:03:08.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb009f8a0 con 0x7f7cb0165550 2024-04-25T19:03:08.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54017070 con 0x7fab4c0053a0 2024-04-25T19:03:08.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54005d00 con 0x7fab4c0053a0 2024-04-25T19:03:08.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54007310 con 0x7fab4c0053a0 2024-04-25T19:03:08.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb009e880 con 0x7f7cb0165550 2024-04-25T19:03:08.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6809da70 con 0x7fab4c0053a0 2024-04-25T19:03:08.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664042020 con 0x7f86740b4b30 2024-04-25T19:03:08.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f0af6000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70015020 con 0x7fdf8006dac0 2024-04-25T19:03:08.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70011890 con 0x7fdf8006dac0 2024-04-25T19:03:08.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011c80 con 0x7fdf8006dac0 2024-04-25T19:03:08.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca00337b0 con 0x7f7cb0165550 2024-04-25T19:03:08.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6809e030 con 0x7fab4c0053a0 2024-04-25T19:03:08.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0af6000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b9070 0x7f0af0161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866403d670 con 0x7f86740b4b30 2024-04-25T19:03:08.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.573+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664037040 con 0x7f86740b4b30 2024-04-25T19:03:08.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0af6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af01601d0 0x7f0af0207a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0aef400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0aef400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47668/0 (socket says 172.21.15.123:47668) 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0aef400640 1 -- 172.21.15.123:0/1755697898 learned_addr learned my addr 172.21.15.123:0/1755697898 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00d5640 con 0x7f0af01601d0 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0aeea00640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b9070 0x7f0af0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0af006cdd0 con 0x7f0af00b9070 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.575+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00b9480 con 0x7f0af006d830 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.575+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf800cf350 con 0x7fdf8006dac0 2024-04-25T19:03:08.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.575+0000 7f0aefe00640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af01601d0 0x7f0af0207a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.575+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf801febf0 con 0x7fdf8006dac0 2024-04-25T19:03:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f0aef400640 1 -- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b9070 msgr2=0x7f0af0161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f0aef400640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b9070 0x7f0af0161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f0aef400640 1 -- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af01601d0 msgr2=0x7f0af0207a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f0aef400640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af01601d0 0x7f0af0207a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0033ba0 con 0x7f7cb0165550 2024-04-25T19:03:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f0aef400640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af01622d0 con 0x7f0af006d830 2024-04-25T19:03:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.574+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86740cc9a0 con 0x7f86740b4b30 2024-04-25T19:03:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86740caf50 con 0x7f86740b4b30 2024-04-25T19:03:08.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.577+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf58005290 con 0x7fdf8006dac0 2024-04-25T19:03:08.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.577+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68076a40 con 0x7fab4c0053a0 2024-04-25T19:03:08.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.577+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c88005290 con 0x7f7cb0165550 2024-04-25T19:03:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.577+0000 7f0aef400640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000efc0 tx=0x7f0ae000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.576+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa298005d10 con 0x7fa2a0071b40 2024-04-25T19:03:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.577+0000 7fa297e00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 0x7fa2800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa298096850 con 0x7fa2a0071b40 2024-04-25T19:03:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018020 con 0x7f0af006d830 2024-04-25T19:03:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae0009e40 con 0x7f0af006d830 2024-04-25T19:03:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7fa29fe00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 0x7fa2800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af01625b0 con 0x7f0af006d830 2024-04-25T19:03:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8674050740 con 0x7f86740b4b30 2024-04-25T19:03:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.578+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af0162b70 con 0x7f0af006d830 2024-04-25T19:03:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.579+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00146a0 con 0x7f0af006d830 2024-04-25T19:03:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.579+0000 7fa29fe00640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 0x7fa2800769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0014f0 tx=0x7fa28c0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.580+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af0076a40 con 0x7f0af006d830 2024-04-25T19:03:08.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.581+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa298022030 con 0x7fa2a0071b40 2024-04-25T19:03:08.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.584+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab5401a070 con 0x7fab4c0053a0 2024-04-25T19:03:08.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.584+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e38137020 con 0x7f2e3017d5a0 2024-04-25T19:03:08.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.585+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ca003e430 con 0x7f7cb0165550 2024-04-25T19:03:08.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.584+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ae0005d10 con 0x7f0af006d830 2024-04-25T19:03:08.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.585+0000 7f0ae7400640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 0x7f0acc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.585+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0ae0096850 con 0x7f0af006d830 2024-04-25T19:03:08.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.586+0000 7fab5f400640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 0x7fab48076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.586+0000 7f0aeea00640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 0x7f0acc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.586+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf70018020 con 0x7fdf8006dac0 2024-04-25T19:03:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.587+0000 7f7cacc00640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 0x7f7c8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.587+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7ca00b6bb0 con 0x7f7cb0165550 2024-04-25T19:03:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.588+0000 7f0aeea00640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 0x7f0acc0769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8034000 tx=0x7f0ad802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.588+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fab54013df0 con 0x7fab4c0053a0 2024-04-25T19:03:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.588+0000 7f7cb4e00640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 0x7f7c8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.589+0000 7f7cb4e00640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 0x7f7c8c0769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c98001800 tx=0x7f7c98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.588+0000 7fab66a00640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 0x7fab48076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.589+0000 7fdf76000640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 0x7fdf60076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.589+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdf70095b40 con 0x7fdf8006dac0 2024-04-25T19:03:08.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.589+0000 7fdf7d400640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 0x7fdf60076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.590+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8664045020 con 0x7f86740b4b30 2024-04-25T19:03:08.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.591+0000 7fdf7d400640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 0x7fdf60076a90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c007fd0 tx=0x7fdf6c007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.591+0000 7fab66a00640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 0x7fab48076a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab680d1ff0 tx=0x7fab60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.592+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab54021020 con 0x7fab4c0053a0 2024-04-25T19:03:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.592+0000 7f866b400640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 0x7f8654076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.592+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f86640b6520 con 0x7f86740b4b30 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.592+0000 7f8672a00640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 0x7f8654076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.593+0000 7f8672a00640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 0x7f8654076ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8674164c10 tx=0x7f8660008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.597+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf7001f020 con 0x7fdf8006dac0 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.601+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ae0022030 con 0x7f0af006d830 2024-04-25T19:03:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.603+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f866404c020 con 0x7f86740b4b30 2024-04-25T19:03:08.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:08.610+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ca004c020 con 0x7f7cb0165550 2024-04-25T19:03:09.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.007+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2860076a40 con 0x7f28600d0be0 2024-04-25T19:03:09.010 INFO:teuthology.orchestra.run.smithi123.stdout:64424509449 2024-04-25T19:03:09.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.008+0000 7f2857e00640 1 -- 172.21.15.123:0/553005192 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f284c0806f0 con 0x7f28600d0be0 2024-04-25T19:03:09.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.011+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 msgr2=0x7f28400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.011+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 0x7f28400769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f286015ec30 tx=0x7f2850009040 comp rx=0 tx=0).stop 2024-04-25T19:03:09.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.011+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 msgr2=0x7f2860160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.011+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f284c009960 tx=0x7f284c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.012+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 shutdown_connections 2024-04-25T19:03:09.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.012+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f28400744b0 0x7f28400769c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.012+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2860165670 0x7f2860070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.012+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2860164b50 0x7f2860073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.012+0000 7f2866a00640 1 --2- 172.21.15.123:0/553005192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28600d0be0 0x7f2860160140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.013+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 >> 172.21.15.123:0/553005192 conn(0x7f28600d6470 msgr2=0x7f28600b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.013+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 shutdown_connections 2024-04-25T19:03:09.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.013+0000 7f2866a00640 1 -- 172.21.15.123:0/553005192 wait complete. 2024-04-25T19:03:09.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.015+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2e14005580 con 0x7f2e3017d5a0 2024-04-25T19:03:09.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.016+0000 7f2e34c00640 1 -- 172.21.15.123:0/623990778 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e380c9020 con 0x7f2e3017d5a0 2024-04-25T19:03:09.022 INFO:teuthology.orchestra.run.smithi123.stdout:64424509449 2024-04-25T19:03:09.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.019+0000 7fa2a6e00640 1 -- 172.21.15.123:0/3892188251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa278005850 con 0x7fa2a0071b40 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 msgr2=0x7f2e18076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 0x7f2e18076bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005f80 tx=0x7f2e2c008040 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 msgr2=0x7f2e300d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 0x7f2e300d7ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e3806b610 tx=0x7f2e38076180 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 shutdown_connections 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2e180746b0 0x7f2e18076bc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e3017d5a0 0x7f2e300d7ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e3003c4e0 0x7f2e3017d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 --2- 172.21.15.123:0/623990778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e30007800 0x7f2e300d6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.020+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 >> 172.21.15.123:0/623990778 conn(0x7f2e30020270 msgr2=0x7f2e30008ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.021+0000 7fa297e00640 1 -- 172.21.15.123:0/3892188251 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa298010040 con 0x7fa2a0071b40 2024-04-25T19:03:09.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.021+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 shutdown_connections 2024-04-25T19:03:09.024 INFO:teuthology.orchestra.run.smithi123.stdout:64424509448 2024-04-25T19:03:09.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.022+0000 7f2e3e600640 1 -- 172.21.15.123:0/623990778 wait complete. 2024-04-25T19:03:09.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.024+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 msgr2=0x7fa2800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.024+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 0x7fa2800769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0014f0 tx=0x7fa28c0078f0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.024+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 msgr2=0x7fa2a015f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.024+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa29800efc0 tx=0x7fa29800c490 comp rx=0 tx=0).stop 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 shutdown_connections 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2800744e0 0x7fa2800769f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a0071b40 0x7fa2a015f040 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a00d4580 0x7fa2a01fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 --2- 172.21.15.123:0/3892188251 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a006c570 0x7fa2a0073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.025+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 >> 172.21.15.123:0/3892188251 conn(0x7fa2a00d6470 msgr2=0x7fa2a00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.026+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 shutdown_connections 2024-04-25T19:03:09.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.026+0000 7fa295600640 1 -- 172.21.15.123:0/3892188251 wait complete. 2024-04-25T19:03:09.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.038+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fab6809d4b0 con 0x7fab4c0053a0 2024-04-25T19:03:09.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.039+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdf58005850 con 0x7fdf8006dac0 2024-04-25T19:03:09.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.039+0000 7fab5f400640 1 -- 172.21.15.123:0/4233266656 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab5405ebc0 con 0x7fab4c0053a0 2024-04-25T19:03:09.041 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:09.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.040+0000 7fdf76000640 1 -- 172.21.15.123:0/3226638386 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf7000d040 con 0x7fdf8006dac0 2024-04-25T19:03:09.042 INFO:teuthology.orchestra.run.smithi123.stdout:60129542155 2024-04-25T19:03:09.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.041+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0af00d3fb0 con 0x7f0af006d830 2024-04-25T19:03:09.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.042+0000 7f0ae7400640 1 -- 172.21.15.123:0/1755697898 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ae0010040 con 0x7f0af006d830 2024-04-25T19:03:09.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.042+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 msgr2=0x7fab48076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.042+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 0x7fab48076a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab680d1ff0 tx=0x7fab60009040 comp rx=0 tx=0).stop 2024-04-25T19:03:09.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.043+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 msgr2=0x7fab68150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.045 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:09.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.043+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c410 tx=0x7fab5400c8d0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 shutdown_connections 2024-04-25T19:03:09.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab48074500 0x7fab48076a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab6809b8b0 0x7fab68166ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6809b4a0 0x7fab68166960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 --2- 172.21.15.123:0/4233266656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab4c0053a0 0x7fab68150c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 >> 172.21.15.123:0/4233266656 conn(0x7fab680d1ae0 msgr2=0x7fab680739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 shutdown_connections 2024-04-25T19:03:09.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fab6e200640 1 -- 172.21.15.123:0/4233266656 wait complete. 2024-04-25T19:03:09.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 msgr2=0x7fdf60076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 0x7fdf60076a90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c007fd0 tx=0x7fdf6c007c20 comp rx=0 tx=0).stop 2024-04-25T19:03:09.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 msgr2=0x7fdf801ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.044+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b440 tx=0x7fdf7000b900 comp rx=0 tx=0).stop 2024-04-25T19:03:09.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 msgr2=0x7f0acc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f86740b87f0 con 0x7f86740b4b30 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 0x7f0acc0769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8034000 tx=0x7f0ad802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 msgr2=0x7f0af0203f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000efc0 tx=0x7f0ae000c490 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 shutdown_connections 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdf60074580 0x7fdf60076a90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf80142c40 0x7fdf801fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8013fbf0 0x7fdf8013c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 --2- 172.21.15.123:0/3226638386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8006dac0 0x7fdf801ff110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 >> 172.21.15.123:0/3226638386 conn(0x7fdf80073200 msgr2=0x7fdf800b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 shutdown_connections 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0acc0744e0 0x7f0acc0769f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af01601d0 0x7f0af0207a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b9070 0x7f0af0161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.047+0000 7f0af6000640 1 --2- 172.21.15.123:0/1755697898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006d830 0x7f0af0203f30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.048+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 >> 172.21.15.123:0/1755697898 conn(0x7f0af00d5d90 msgr2=0x7f0af00d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.048+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 shutdown_connections 2024-04-25T19:03:09.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.048+0000 7fdf7f200640 1 -- 172.21.15.123:0/3226638386 wait complete. 2024-04-25T19:03:09.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.048+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 shutdown_connections 2024-04-25T19:03:09.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.048+0000 7f0af6000640 1 -- 172.21.15.123:0/1755697898 wait complete. 2024-04-25T19:03:09.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.050+0000 7f866b400640 1 -- 172.21.15.123:0/3663947815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8664080950 con 0x7f86740b4b30 2024-04-25T19:03:09.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.050+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7c88005850 con 0x7f7cb0165550 2024-04-25T19:03:09.052 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:09.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.1 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f7cacc00640 1 -- 172.21.15.123:0/1119493321 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ca0080ff0 con 0x7f7cb0165550 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 msgr2=0x7f8654076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 0x7f8654076ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8674164c10 tx=0x7f8660008040 comp rx=0 tx=0).stop 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 msgr2=0x7f8674163d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f866402a6e0 tx=0x7f86640046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.056 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 shutdown_connections 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f86540745b0 0x7f8654076ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8674160070 0x7f86740cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86740b9410 0x7f86740cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 --2- 172.21.15.123:0/3663947815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86740b4b30 0x7f8674163d70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 >> 172.21.15.123:0/3663947815 conn(0x7f86740d65d0 msgr2=0x7f867415ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.054+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 shutdown_connections 2024-04-25T19:03:09.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.055+0000 7f8679000640 1 -- 172.21.15.123:0/3663947815 wait complete. 2024-04-25T19:03:09.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 msgr2=0x7f7c8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 0x7f7c8c0769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c98001800 tx=0x7f7c98009040 comp rx=0 tx=0).stop 2024-04-25T19:03:09.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 msgr2=0x7f7cb0152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0152f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0033000 tx=0x7f7ca00046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 shutdown_connections 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7c8c0744b0 0x7f7c8c0769c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0165550 0x7f7cb0152f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb015c670 0x7f7cb009f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 --2- 172.21.15.123:0/1119493321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb00b1990 0x7f7cb01fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:09.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.058+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 >> 172.21.15.123:0/1119493321 conn(0x7f7cb0073ce0 msgr2=0x7f7cb0151720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:09.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.059+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 shutdown_connections 2024-04-25T19:03:09.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:09.059+0000 7f7cb7800640 1 -- 172.21.15.123:0/1119493321 wait complete. 2024-04-25T19:03:09.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.2 2024-04-25T19:03:09.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.0 2024-04-25T19:03:09.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.4 2024-04-25T19:03:09.077 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:09.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-04-25T19:03:09.083 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:09.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-04-25T19:03:09.086 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:09.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.6 2024-04-25T19:03:09.091 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:09.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-04-25T19:03:09.094 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:10.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T19:03:10.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T19:03:10.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T19:03:10.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.326+0000 7fce9f800640 1 Processor -- start 2024-04-25T19:03:10.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.326+0000 7fce9f800640 1 -- start start 2024-04-25T19:03:10.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.329+0000 7fce9f800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce98161b10 0x7fce98161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.329+0000 7fce9f800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce98071600 con 0x7fce98154950 2024-04-25T19:03:10.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.329+0000 7fce9f800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce98162460 con 0x7fce980d1120 2024-04-25T19:03:10.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.329+0000 7fce9f800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fce980d1530 con 0x7fce98161b10 2024-04-25T19:03:10.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce97e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce98161b10 0x7fce98161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce9d800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fce98154950 0x7fce98154d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45324/0 (socket says 172.21.15.123:45324) 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce9d800640 1 -- 172.21.15.123:0/108077026 learned_addr learned my addr 172.21.15.123:0/108077026 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce97e00640 1 -- 172.21.15.123:0/108077026 >> v1:172.21.15.146:6789/0 conn(0x7fce980d1120 legacy=0x7fce98070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce97e00640 1 -- 172.21.15.123:0/108077026 >> v1:172.21.15.123:6789/0 conn(0x7fce98154950 legacy=0x7fce98154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce97e00640 1 -- 172.21.15.123:0/108077026 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce980aec30 con 0x7fce98161b10 2024-04-25T19:03:10.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.330+0000 7fce97400640 1 -- 172.21.15.123:0/108077026 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096875899 0 0) 0x7fce98071600 con 0x7fce98154950 2024-04-25T19:03:10.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.331+0000 7fce97e00640 1 --2- 172.21.15.123:0/108077026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98161b10 0x7fce98161f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce840097a0 tx=0x7fce8402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1360c0ee1cb1e453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.332+0000 7fce97400640 1 -- 172.21.15.123:0/108077026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84037070 con 0x7fce98161b10 2024-04-25T19:03:10.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.332+0000 7fce97400640 1 -- 172.21.15.123:0/108077026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce84032440 con 0x7fce98161b10 2024-04-25T19:03:10.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.332+0000 7fce97400640 1 -- 172.21.15.123:0/108077026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8402ae60 con 0x7fce98161b10 2024-04-25T19:03:10.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 -- 172.21.15.123:0/108077026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98161b10 msgr2=0x7fce98161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 --2- 172.21.15.123:0/108077026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98161b10 0x7fce98161f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce840097a0 tx=0x7fce8402a110 comp rx=0 tx=0).stop 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 -- 172.21.15.123:0/108077026 shutdown_connections 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 --2- 172.21.15.123:0/108077026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98161b10 0x7fce98161f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 -- 172.21.15.123:0/108077026 >> 172.21.15.123:0/108077026 conn(0x7fce980d1ae0 msgr2=0x7fce980d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 -- 172.21.15.123:0/108077026 shutdown_connections 2024-04-25T19:03:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.333+0000 7fce9f800640 1 -- 172.21.15.123:0/108077026 wait complete. 2024-04-25T19:03:10.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.335+0000 7fce9f800640 1 Processor -- start 2024-04-25T19:03:10.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.336+0000 7fce9f800640 1 -- start start 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 0x7fce980b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 0x7fce981f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce980af4d0 con 0x7fce98154950 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fce980af1f0 con 0x7fce98161b10 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce9f800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce980ae7e0 con 0x7fce980d1120 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.337+0000 7fce97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34438/0 (socket says 172.21.15.123:34438) 2024-04-25T19:03:10.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 -- 172.21.15.123:0/493366756 learned_addr learned my addr 172.21.15.123:0/493366756 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce9d800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 0x7fce981f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce9ce00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 0x7fce980b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 -- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 msgr2=0x7fce980b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 0x7fce980b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 -- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 msgr2=0x7fce981f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 0x7fce981f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 -- 172.21.15.123:0/493366756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce981f9ec0 con 0x7fce98154950 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce9d800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 0x7fce981f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce9ce00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 0x7fce980b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.338+0000 7fce97e00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce840097a0 tx=0x7fce840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.339+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84037050 con 0x7fce98154950 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.339+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce84038040 con 0x7fce98154950 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.339+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84004490 con 0x7fce98154950 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.339+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce981fa1a0 con 0x7fce98154950 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.339+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce981fa760 con 0x7fce98154950 2024-04-25T19:03:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.340+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce98076a40 con 0x7fce98154950 2024-04-25T19:03:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.341+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce84030900 con 0x7fce98154950 2024-04-25T19:03:10.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.342+0000 7fce94c00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 0x7fce780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.342+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fce840b6210 con 0x7fce98154950 2024-04-25T19:03:10.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.343+0000 7fce9ce00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 0x7fce780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.343+0000 7fce9ce00640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 0x7fce780769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000f60 tx=0x7fce8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.348+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce840805c0 con 0x7fce98154950 2024-04-25T19:03:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.350+0000 7fab86200640 1 Processor -- start 2024-04-25T19:03:10.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.350+0000 7fab86200640 1 -- start start 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.353+0000 7fab86200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab800ae7e0 0x7fab800aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.353+0000 7fab86200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab800b4750 con 0x7fab800b57f0 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.353+0000 7fab86200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab800b4a30 con 0x7fab8015df00 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.353+0000 7fab86200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fab8015e310 con 0x7fab800ae7e0 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.353+0000 7fab7ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fab800b57f0 0x7fab800b5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45328/0 (socket says 172.21.15.123:45328) 2024-04-25T19:03:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7ea00640 1 -- 172.21.15.123:0/4215262785 learned_addr learned my addr 172.21.15.123:0/4215262785 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7fe00640 1 --2- 172.21.15.123:0/4215262785 >> v2:172.21.15.146:3300/0 conn(0x7fab800ae7e0 0x7fab800aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509607178 0 0) 0x7fab800b4750 con 0x7fab800b57f0 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab64007930 con 0x7fab800b57f0 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036309865 0 0) 0x7fab64007930 con 0x7fab800b57f0 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 >> v2:172.21.15.146:3300/0 conn(0x7fab800ae7e0 msgr2=0x7fab800aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 --2- 172.21.15.123:0/4215262785 >> v2:172.21.15.146:3300/0 conn(0x7fab800ae7e0 0x7fab800aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 >> v1:172.21.15.146:6789/0 conn(0x7fab8015df00 legacy=0x7fab8006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab800b4d10 con 0x7fab800b57f0 2024-04-25T19:03:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.354+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fab6c001670 con 0x7fab800b57f0 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 >> v1:172.21.15.123:6789/0 conn(0x7fab800b57f0 legacy=0x7fab800b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64007dc0 0x7fab640081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 0x7fab64009020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64009680 0x7fab6400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab8015e310 con 0x7fab64008c10 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab640105c0 con 0x7fab64007dc0 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab640107c0 con 0x7fab64009680 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7ea00640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 0x7fab64009020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.355+0000 7fab7f400640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64007dc0 0x7fab640081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7fe00640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64009680 0x7fab6400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7ea00640 1 -- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64009680 msgr2=0x7fab6400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7ea00640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64009680 0x7fab6400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7ea00640 1 -- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64007dc0 msgr2=0x7fab640081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7ea00640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64007dc0 0x7fab640081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7ea00640 1 -- 172.21.15.123:0/4215262785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64007930 con 0x7fab64008c10 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.356+0000 7fab7fe00640 1 -- 172.21.15.123:0/4215262785 reap_dead start 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.357+0000 7fb6cc400640 1 Processor -- start 2024-04-25T19:03:10.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.357+0000 7fb6cc400640 1 -- start start 2024-04-25T19:03:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.357+0000 7fab7ea00640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 0x7fab64009020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab6c004ff0 tx=0x7fab64009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfe052d941674da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.358+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c004bc0 con 0x7fab64008c10 2024-04-25T19:03:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.358+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c02f030 con 0x7fab64008c10 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.358+0000 7fab7e000640 1 -- 172.21.15.123:0/4215262785 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c005d10 con 0x7fab64008c10 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 -- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 msgr2=0x7fab64009020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 0x7fab64009020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab6c004ff0 tx=0x7fab64009c40 comp rx=0 tx=0).stop 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 -- 172.21.15.123:0/4215262785 shutdown_connections 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 --2- 172.21.15.123:0/4215262785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64008c10 0x7fab64009020 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 -- 172.21.15.123:0/4215262785 >> 172.21.15.123:0/4215262785 conn(0x7fab800d1ae0 msgr2=0x7fab800d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 -- 172.21.15.123:0/4215262785 shutdown_connections 2024-04-25T19:03:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fab86200640 1 -- 172.21.15.123:0/4215262785 wait complete. 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6cc400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6c415df00 0x7fb6c406f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6cc400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c40b5f00 0x7fb6c40ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6cc400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c40b4750 con 0x7fb6c406f780 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6cc400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c415e310 con 0x7fb6c415df00 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6cc400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c40b5360 con 0x7fb6c40b5f00 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.359+0000 7fb6c9a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6c415df00 0x7fb6c406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6c415df00 0x7fb6c406f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34454/0 (socket says 172.21.15.123:34454) 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 -- 172.21.15.123:0/1361887097 learned_addr learned my addr 172.21.15.123:0/1361887097 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 -- 172.21.15.123:0/1361887097 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c40b5f00 msgr2=0x7fb6c40ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 --2- 172.21.15.123:0/1361887097 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c40b5f00 0x7fb6c40ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 -- 172.21.15.123:0/1361887097 >> v1:172.21.15.146:6789/0 conn(0x7fb6c406f780 legacy=0x7fb6c40b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.360+0000 7fb6c9a00640 1 -- 172.21.15.123:0/1361887097 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c40b4a30 con 0x7fb6c415df00 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.361+0000 7fb6c9a00640 1 --2- 172.21.15.123:0/1361887097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c415df00 0x7fb6c406f240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6c406c680 tx=0x7fb6b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266203ad403741ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.361+0000 7fb6c3e00640 1 -- 172.21.15.123:0/1361887097 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8037070 con 0x7fb6c415df00 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.361+0000 7fb6c3e00640 1 -- 172.21.15.123:0/1361887097 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b8032440 con 0x7fb6c415df00 2024-04-25T19:03:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.361+0000 7fb6c3e00640 1 -- 172.21.15.123:0/1361887097 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8032810 con 0x7fb6c415df00 2024-04-25T19:03:10.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.361+0000 7fab86200640 1 Processor -- start 2024-04-25T19:03:10.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.362+0000 7fab86200640 1 -- start start 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.362+0000 7fb6cc400640 1 -- 172.21.15.123:0/1361887097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c415df00 msgr2=0x7fb6c406f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.362+0000 7fb6cc400640 1 --2- 172.21.15.123:0/1361887097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c415df00 0x7fb6c406f240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6c406c680 tx=0x7fb6b802a180 comp rx=0 tx=0).stop 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 -- 172.21.15.123:0/1361887097 shutdown_connections 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 --2- 172.21.15.123:0/1361887097 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c40b5f00 0x7fb6c40ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 --2- 172.21.15.123:0/1361887097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c415df00 0x7fb6c406f240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 -- 172.21.15.123:0/1361887097 >> 172.21.15.123:0/1361887097 conn(0x7fb6c40d1ae0 msgr2=0x7fb6c40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 -- 172.21.15.123:0/1361887097 shutdown_connections 2024-04-25T19:03:10.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fb6cc400640 1 -- 172.21.15.123:0/1361887097 wait complete. 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64008c10 0x7fab801f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8013ca00 0x7fab801f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b5c00 con 0x7fab80155890 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b5360 con 0x7fab64008c10 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab86200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab800b5d80 con 0x7fab8013ca00 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.363+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34466/0 (socket says 172.21.15.123:34466) 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 -- 172.21.15.123:0/3140862859 learned_addr learned my addr 172.21.15.123:0/3140862859 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7f400640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64008c10 0x7fab801f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7ea00640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8013ca00 0x7fab801f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 -- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8013ca00 msgr2=0x7fab801f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8013ca00 0x7fab801f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 -- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64008c10 msgr2=0x7fab801f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64008c10 0x7fab801f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 -- 172.21.15.123:0/3140862859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8015a7b0 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.364+0000 7fab7fe00640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab80160830 tx=0x7fab78009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78007cb0 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7801b070 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801f6640 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78016670 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801f2580 con 0x7fab80155890 2024-04-25T19:03:10.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.365+0000 7fb6cc400640 1 Processor -- start 2024-04-25T19:03:10.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.366+0000 7fb6cc400640 1 -- start start 2024-04-25T19:03:10.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.366+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab801f63b0 con 0x7fab80155890 2024-04-25T19:03:10.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c415df00 0x7fb6c4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40aed20 con 0x7fb6c40b5f00 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40b4ff0 con 0x7fb6c406f780 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fb6cc400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c40aeea0 con 0x7fb6c415df00 2024-04-25T19:03:10.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.367+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab78005d10 con 0x7fab80155890 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6c9a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6c9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6c9a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:48510/0 (socket says 172.21.15.123:48510) 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6c9a00640 1 -- 172.21.15.123:0/3076713182 learned_addr learned my addr 172.21.15.123:0/3076713182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6c9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34482/0 (socket says 172.21.15.123:34482) 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.368+0000 7fb6ca400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c415df00 0x7fb6c4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 -- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c415df00 msgr2=0x7fb6c4159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fab77400640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 0x7fab60076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c415df00 0x7fb6c4159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 -- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 msgr2=0x7fb6c41f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 -- 172.21.15.123:0/3076713182 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c40cb980 con 0x7fb6c406f780 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9000640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fab78095da0 con 0x7fab80155890 2024-04-25T19:03:10.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fab7f400640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 0x7fab60076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.369+0000 7fb6c9a00640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80090b0 tx=0x7fb6b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8041070 con 0x7fb6c406f780 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b803c870 con 0x7fb6c406f780 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8037040 con 0x7fb6c406f780 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c40ccb60 con 0x7fb6c406f780 2024-04-25T19:03:10.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c40cd120 con 0x7fb6c406f780 2024-04-25T19:03:10.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.370+0000 7fab7f400640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 0x7fab60076a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab80069e80 tx=0x7fab70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.371+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4076a40 con 0x7fb6c406f780 2024-04-25T19:03:10.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.374+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab78025020 con 0x7fab80155890 2024-04-25T19:03:10.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.375+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb6b8044070 con 0x7fb6c406f780 2024-04-25T19:03:10.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.376+0000 7fb6c1600640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.376+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb6b80b6560 con 0x7fb6c406f780 2024-04-25T19:03:10.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.376+0000 7fb6c9000640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.377+0000 7fb6c9000640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 0x7fb6a40769c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6c415c990 tx=0x7fb6b0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.390+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6b80802c0 con 0x7fb6c406f780 2024-04-25T19:03:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.552+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fce981f7410 con 0x7fce98154950 2024-04-25T19:03:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.553+0000 7fce94c00640 1 -- 172.21.15.123:0/493366756 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce8407fec0 con 0x7fce98154950 2024-04-25T19:03:10.555 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 msgr2=0x7fce780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 0x7fce780769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000f60 tx=0x7fce8c009040 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 msgr2=0x7fce981f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce840097a0 tx=0x7fce840046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 shutdown_connections 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fce780744b0 0x7fce780769c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce98161b10 0x7fce981f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce98154950 0x7fce981f6960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 --2- 172.21.15.123:0/493366756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce980d1120 0x7fce980b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 >> 172.21.15.123:0/493366756 conn(0x7fce980d1ae0 msgr2=0x7fce980b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 shutdown_connections 2024-04-25T19:03:10.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.555+0000 7fce9f800640 1 -- 172.21.15.123:0/493366756 wait complete. 2024-04-25T19:03:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.568+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab80076a40 con 0x7fab80155890 2024-04-25T19:03:10.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.569+0000 7fab77400640 1 -- 172.21.15.123:0/3140862859 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab8015a7b0 con 0x7fab80155890 2024-04-25T19:03:10.571 INFO:teuthology.orchestra.run.smithi123.stdout:64424509451 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 msgr2=0x7fab60076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 0x7fab60076a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab80069e80 tx=0x7fab70009040 comp rx=0 tx=0).stop 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 msgr2=0x7fab80155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab80160830 tx=0x7fab78009f70 comp rx=0 tx=0).stop 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 shutdown_connections 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab60074500 0x7fab60076a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab80155890 0x7fab80155ca0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8013ca00 0x7fab801f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 --2- 172.21.15.123:0/3140862859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab64008c10 0x7fab801f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 >> 172.21.15.123:0/3140862859 conn(0x7fab800d1ae0 msgr2=0x7fab80071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 shutdown_connections 2024-04-25T19:03:10.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.571+0000 7fab86200640 1 -- 172.21.15.123:0/3140862859 wait complete. 2024-04-25T19:03:10.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-04-25T19:03:10.583 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:10.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.585+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb6c415d4b0 con 0x7fb6c406f780 2024-04-25T19:03:10.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.586+0000 7fb6c1600640 1 -- 172.21.15.123:0/3076713182 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6b80845c0 con 0x7fb6c406f780 2024-04-25T19:03:10.588 INFO:teuthology.orchestra.run.smithi123.stdout:64424509450 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 msgr2=0x7fb6a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 0x7fb6a40769c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6c415c990 tx=0x7fb6b0009060 comp rx=0 tx=0).stop 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 msgr2=0x7fb6c40cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80090b0 tx=0x7fb6b80046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 shutdown_connections 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6c415df00 0x7fb6c4159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6c40b5f00 0x7fb6c41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 --2- 172.21.15.123:0/3076713182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6c406f780 0x7fb6c40cb440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 >> 172.21.15.123:0/3076713182 conn(0x7fb6c40d1ae0 msgr2=0x7fb6c406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 shutdown_connections 2024-04-25T19:03:10.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.589+0000 7fb6cc400640 1 -- 172.21.15.123:0/3076713182 wait complete. 2024-04-25T19:03:10.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-04-25T19:03:10.601 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:10.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-04-25T19:03:10.618 DEBUG:teuthology.parallel:result is None 2024-04-25T19:03:10.618 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T19:03:10.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:03:10.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.915+0000 7f0af8800640 1 Processor -- start 2024-04-25T19:03:10.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.916+0000 7f0af8800640 1 -- start start 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0af015df00 0x7f0af006f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af00b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0af00b5d30 0x7f0af00ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0af015e310 con 0x7f0af00b5d30 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0af00b5360 con 0x7f0af015df00 2024-04-25T19:03:10.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af8800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00b6140 con 0x7f0af006f780 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af6800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0af00b5d30 0x7f0af00ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af6800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0af00b5d30 0x7f0af00ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34488/0 (socket says 172.21.15.123:34488) 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af6800640 1 -- 172.21.15.123:0/2301520294 learned_addr learned my addr 172.21.15.123:0/2301520294 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.918+0000 7f0af5e00640 1 --2- 172.21.15.123:0/2301520294 >> v2:172.21.15.146:3300/0 conn(0x7f0af015df00 0x7f0af006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af5400640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af00b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af6800640 1 -- 172.21.15.123:0/2301520294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 msgr2=0x7f0af00b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af6800640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af00b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af6800640 1 -- 172.21.15.123:0/2301520294 >> v2:172.21.15.146:3300/0 conn(0x7f0af015df00 msgr2=0x7f0af006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af6800640 1 --2- 172.21.15.123:0/2301520294 >> v2:172.21.15.146:3300/0 conn(0x7f0af015df00 0x7f0af006f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.919+0000 7f0af6800640 1 -- 172.21.15.123:0/2301520294 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af00b4750 con 0x7f0af00b5d30 2024-04-25T19:03:10.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.920+0000 7f0af6800640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00b5d30 0x7f0af00ae7e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0af006c680 tx=0x7f0aec02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd6a4b0ec53f676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.920+0000 7f0af4a00640 1 -- 172.21.15.123:0/2301520294 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec036070 con 0x7f0af00b5d30 2024-04-25T19:03:10.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.920+0000 7f0af4a00640 1 -- 172.21.15.123:0/2301520294 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec031600 con 0x7f0af00b5d30 2024-04-25T19:03:10.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.920+0000 7f0af4a00640 1 -- 172.21.15.123:0/2301520294 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0319d0 con 0x7f0af00b5d30 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 -- 172.21.15.123:0/2301520294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00b5d30 msgr2=0x7f0af00ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00b5d30 0x7f0af00ae7e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0af006c680 tx=0x7f0aec02a610 comp rx=0 tx=0).stop 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 -- 172.21.15.123:0/2301520294 shutdown_connections 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00b5d30 0x7f0af00ae7e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 --2- 172.21.15.123:0/2301520294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af00b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 --2- 172.21.15.123:0/2301520294 >> v2:172.21.15.146:3300/0 conn(0x7f0af015df00 0x7f0af006f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 -- 172.21.15.123:0/2301520294 >> 172.21.15.123:0/2301520294 conn(0x7f0af00d1ae0 msgr2=0x7f0af00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 -- 172.21.15.123:0/2301520294 shutdown_connections 2024-04-25T19:03:10.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.921+0000 7f0af8800640 1 -- 172.21.15.123:0/2301520294 wait complete. 2024-04-25T19:03:10.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.923+0000 7f0af8800640 1 Processor -- start 2024-04-25T19:03:10.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.923+0000 7f0af8800640 1 -- start start 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b5d30 0x7f0af0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00b4ff0 con 0x7f0af015df00 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00b4d10 con 0x7f0af00b5d30 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af8800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00aed20 con 0x7f0af006f780 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34504/0 (socket says 172.21.15.123:34504) 2024-04-25T19:03:10.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af6800640 1 -- 172.21.15.123:0/3811927260 learned_addr learned my addr 172.21.15.123:0/3811927260 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:10.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af5400640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b5d30 0x7f0af0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.925+0000 7f0af5e00640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 -- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 msgr2=0x7f0af0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af0158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 -- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b5d30 msgr2=0x7f0af0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b5d30 0x7f0af0157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 -- 172.21.15.123:0/3811927260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af01577d0 con 0x7f0af015df00 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af6800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f40 tx=0x7f0aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03f020 con 0x7f0af015df00 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec037040 con 0x7f0af015df00 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec036070 con 0x7f0af015df00 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af014a0a0 con 0x7f0af015df00 2024-04-25T19:03:10.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.926+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af014a660 con 0x7f0af015df00 2024-04-25T19:03:10.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.927+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af0149e10 con 0x7f0af015df00 2024-04-25T19:03:10.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.928+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0aec042020 con 0x7f0af015df00 2024-04-25T19:03:10.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.929+0000 7f0aea000640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 0x7f0ad40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.930+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0aec0b5dc0 con 0x7f0af015df00 2024-04-25T19:03:10.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.930+0000 7f0af5e00640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 0x7f0ad40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.930+0000 7f0af5e00640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 0x7f0ad40769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006fd0 tx=0x7f0ae4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:10.934+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0aec049020 con 0x7f0af015df00 2024-04-25T19:03:11.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.121+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0af00cd810 con 0x7f0ad40744b0 2024-04-25T19:03:11.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.148+0000 7f0aea000640 1 -- 172.21.15.123:0/3811927260 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332522 (secure 0 0 0) 0x7f0af00cd810 con 0x7f0ad40744b0 2024-04-25T19:03:11.154 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:11.156 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:03:11.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.155+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 msgr2=0x7f0ad40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.155+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 0x7f0ad40769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006fd0 tx=0x7f0ae4008040 comp rx=0 tx=0).stop 2024-04-25T19:03:11.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.155+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 msgr2=0x7f0af00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f40 tx=0x7f0aec004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:11.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 shutdown_connections 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0ad40744b0 0x7f0ad40769c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af015df00 0x7f0af00b1700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0af00b5d30 0x7f0af0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 --2- 172.21.15.123:0/3811927260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0af006f780 0x7f0af0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 >> 172.21.15.123:0/3811927260 conn(0x7f0af00d1ae0 msgr2=0x7f0af01539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 shutdown_connections 2024-04-25T19:03:11.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.156+0000 7f0af8800640 1 -- 172.21.15.123:0/3811927260 wait complete. 2024-04-25T19:03:11.184 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T19:03:10.973164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219516,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755204,"statfs":{"total":773094113280,"available":772869328896,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005398"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137574+0000","last_change":"2024-04-25T19:02:39.052814+0000","last_active":"2024-04-25T19:02:45.137574+0000","last_peered":"2024-04-25T19:02:45.137574+0000","last_clean":"2024-04-25T19:02:45.137574+0000","last_became_active":"2024-04-25T19:02:39.052399+0000","last_became_peered":"2024-04-25T19:02:39.052399+0000","last_unstale":"2024-04-25T19:02:45.137574+0000","last_undegraded":"2024-04-25T19:02:45.137574+0000","last_fullsized":"2024-04-25T19:02:45.137574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202717+0000","last_change":"2024-04-25T19:02:40.072630+0000","last_active":"2024-04-25T19:02:45.202717+0000","last_peered":"2024-04-25T19:02:45.202717+0000","last_clean":"2024-04-25T19:02:45.202717+0000","last_became_active":"2024-04-25T19:02:40.072046+0000","last_became_peered":"2024-04-25T19:02:40.072046+0000","last_unstale":"2024-04-25T19:02:45.202717+0000","last_undegraded":"2024-04-25T19:02:45.202717+0000","last_fullsized":"2024-04-25T19:02:45.202717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200723+0000","last_change":"2024-04-25T19:02:44.161105+0000","last_active":"2024-04-25T19:02:45.200723+0000","last_peered":"2024-04-25T19:02:45.200723+0000","last_clean":"2024-04-25T19:02:45.200723+0000","last_became_active":"2024-04-25T19:02:44.160613+0000","last_became_peered":"2024-04-25T19:02:44.160613+0000","last_unstale":"2024-04-25T19:02:45.200723+0000","last_undegraded":"2024-04-25T19:02:45.200723+0000","last_fullsized":"2024-04-25T19:02:45.200723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:38.319660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203032+0000","last_change":"2024-04-25T19:02:39.042391+0000","last_active":"2024-04-25T19:02:45.203032+0000","last_peered":"2024-04-25T19:02:45.203032+0000","last_clean":"2024-04-25T19:02:45.203032+0000","last_became_active":"2024-04-25T19:02:39.042017+0000","last_became_peered":"2024-04-25T19:02:39.042017+0000","last_unstale":"2024-04-25T19:02:45.203032+0000","last_undegraded":"2024-04-25T19:02:45.203032+0000","last_fullsized":"2024-04-25T19:02:45.203032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203099+0000","last_change":"2024-04-25T19:02:40.073983+0000","last_active":"2024-04-25T19:02:45.203099+0000","last_peered":"2024-04-25T19:02:45.203099+0000","last_clean":"2024-04-25T19:02:45.203099+0000","last_became_active":"2024-04-25T19:02:40.069080+0000","last_became_peered":"2024-04-25T19:02:40.069080+0000","last_unstale":"2024-04-25T19:02:45.203099+0000","last_undegraded":"2024-04-25T19:02:45.203099+0000","last_fullsized":"2024-04-25T19:02:45.203099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198316+0000","last_change":"2024-04-25T19:02:44.148887+0000","last_active":"2024-04-25T19:02:45.198316+0000","last_peered":"2024-04-25T19:02:45.198316+0000","last_clean":"2024-04-25T19:02:45.198316+0000","last_became_active":"2024-04-25T19:02:44.148199+0000","last_became_peered":"2024-04-25T19:02:44.148199+0000","last_unstale":"2024-04-25T19:02:45.198316+0000","last_undegraded":"2024-04-25T19:02:45.198316+0000","last_fullsized":"2024-04-25T19:02:45.198316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:55.854080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199269+0000","last_change":"2024-04-25T19:02:39.047541+0000","last_active":"2024-04-25T19:02:45.199269+0000","last_peered":"2024-04-25T19:02:45.199269+0000","last_clean":"2024-04-25T19:02:45.199269+0000","last_became_active":"2024-04-25T19:02:39.047208+0000","last_became_peered":"2024-04-25T19:02:39.047208+0000","last_unstale":"2024-04-25T19:02:45.199269+0000","last_undegraded":"2024-04-25T19:02:45.199269+0000","last_fullsized":"2024-04-25T19:02:45.199269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200023+0000","last_change":"2024-04-25T19:02:40.076534+0000","last_active":"2024-04-25T19:02:45.200023+0000","last_peered":"2024-04-25T19:02:45.200023+0000","last_clean":"2024-04-25T19:02:45.200023+0000","last_became_active":"2024-04-25T19:02:40.076129+0000","last_became_peered":"2024-04-25T19:02:40.076129+0000","last_unstale":"2024-04-25T19:02:45.200023+0000","last_undegraded":"2024-04-25T19:02:45.200023+0000","last_fullsized":"2024-04-25T19:02:45.200023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198683+0000","last_change":"2024-04-25T19:02:44.153635+0000","last_active":"2024-04-25T19:02:45.198683+0000","last_peered":"2024-04-25T19:02:45.198683+0000","last_clean":"2024-04-25T19:02:45.198683+0000","last_became_active":"2024-04-25T19:02:44.153134+0000","last_became_peered":"2024-04-25T19:02:44.153134+0000","last_unstale":"2024-04-25T19:02:45.198683+0000","last_undegraded":"2024-04-25T19:02:45.198683+0000","last_fullsized":"2024-04-25T19:02:45.198683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:25.102590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199200+0000","last_change":"2024-04-25T19:02:39.047906+0000","last_active":"2024-04-25T19:02:45.199200+0000","last_peered":"2024-04-25T19:02:45.199200+0000","last_clean":"2024-04-25T19:02:45.199200+0000","last_became_active":"2024-04-25T19:02:39.044722+0000","last_became_peered":"2024-04-25T19:02:39.044722+0000","last_unstale":"2024-04-25T19:02:45.199200+0000","last_undegraded":"2024-04-25T19:02:45.199200+0000","last_fullsized":"2024-04-25T19:02:45.199200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130532+0000","last_change":"2024-04-25T19:02:40.084937+0000","last_active":"2024-04-25T19:02:45.130532+0000","last_peered":"2024-04-25T19:02:45.130532+0000","last_clean":"2024-04-25T19:02:45.130532+0000","last_became_active":"2024-04-25T19:02:40.084547+0000","last_became_peered":"2024-04-25T19:02:40.084547+0000","last_unstale":"2024-04-25T19:02:45.130532+0000","last_undegraded":"2024-04-25T19:02:45.130532+0000","last_fullsized":"2024-04-25T19:02:45.130532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:00.967270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197619+0000","last_change":"2024-04-25T19:02:44.152662+0000","last_active":"2024-04-25T19:02:45.197619+0000","last_peered":"2024-04-25T19:02:45.197619+0000","last_clean":"2024-04-25T19:02:45.197619+0000","last_became_active":"2024-04-25T19:02:44.152150+0000","last_became_peered":"2024-04-25T19:02:44.152150+0000","last_unstale":"2024-04-25T19:02:45.197619+0000","last_undegraded":"2024-04-25T19:02:45.197619+0000","last_fullsized":"2024-04-25T19:02:45.197619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:04.766506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129234+0000","last_change":"2024-04-25T19:02:40.069483+0000","last_active":"2024-04-25T19:02:45.129234+0000","last_peered":"2024-04-25T19:02:45.129234+0000","last_clean":"2024-04-25T19:02:45.129234+0000","last_became_active":"2024-04-25T19:02:40.068633+0000","last_became_peered":"2024-04-25T19:02:40.068633+0000","last_unstale":"2024-04-25T19:02:45.129234+0000","last_undegraded":"2024-04-25T19:02:45.129234+0000","last_fullsized":"2024-04-25T19:02:45.129234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:46.741238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130329+0000","last_change":"2024-04-25T19:02:39.036777+0000","last_active":"2024-04-25T19:02:45.130329+0000","last_peered":"2024-04-25T19:02:45.130329+0000","last_clean":"2024-04-25T19:02:45.130329+0000","last_became_active":"2024-04-25T19:02:39.036471+0000","last_became_peered":"2024-04-25T19:02:39.036471+0000","last_unstale":"2024-04-25T19:02:45.130329+0000","last_undegraded":"2024-04-25T19:02:45.130329+0000","last_fullsized":"2024-04-25T19:02:45.130329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:41.460879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130171+0000","last_change":"2024-04-25T19:02:44.153484+0000","last_active":"2024-04-25T19:02:45.130171+0000","last_peered":"2024-04-25T19:02:45.130171+0000","last_clean":"2024-04-25T19:02:45.130171+0000","last_became_active":"2024-04-25T19:02:44.153061+0000","last_became_peered":"2024-04-25T19:02:44.153061+0000","last_unstale":"2024-04-25T19:02:45.130171+0000","last_undegraded":"2024-04-25T19:02:45.130171+0000","last_fullsized":"2024-04-25T19:02:45.130171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:14.106695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201441+0000","last_change":"2024-04-25T19:02:40.071684+0000","last_active":"2024-04-25T19:02:45.201441+0000","last_peered":"2024-04-25T19:02:45.201441+0000","last_clean":"2024-04-25T19:02:45.201441+0000","last_became_active":"2024-04-25T19:02:40.071149+0000","last_became_peered":"2024-04-25T19:02:40.071149+0000","last_unstale":"2024-04-25T19:02:45.201441+0000","last_undegraded":"2024-04-25T19:02:45.201441+0000","last_fullsized":"2024-04-25T19:02:45.201441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134616+0000","last_change":"2024-04-25T19:02:39.047265+0000","last_active":"2024-04-25T19:02:45.134616+0000","last_peered":"2024-04-25T19:02:45.134616+0000","last_clean":"2024-04-25T19:02:45.134616+0000","last_became_active":"2024-04-25T19:02:39.046336+0000","last_became_peered":"2024-04-25T19:02:39.046336+0000","last_unstale":"2024-04-25T19:02:45.134616+0000","last_undegraded":"2024-04-25T19:02:45.134616+0000","last_fullsized":"2024-04-25T19:02:45.134616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:02.247449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198218+0000","last_change":"2024-04-25T19:02:44.139798+0000","last_active":"2024-04-25T19:02:45.198218+0000","last_peered":"2024-04-25T19:02:45.198218+0000","last_clean":"2024-04-25T19:02:45.198218+0000","last_became_active":"2024-04-25T19:02:44.139333+0000","last_became_peered":"2024-04-25T19:02:44.139333+0000","last_unstale":"2024-04-25T19:02:45.198218+0000","last_undegraded":"2024-04-25T19:02:45.198218+0000","last_fullsized":"2024-04-25T19:02:45.198218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:05.053484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131214+0000","last_change":"2024-04-25T19:02:40.065574+0000","last_active":"2024-04-25T19:02:45.131214+0000","last_peered":"2024-04-25T19:02:45.131214+0000","last_clean":"2024-04-25T19:02:45.131214+0000","last_became_active":"2024-04-25T19:02:40.060953+0000","last_became_peered":"2024-04-25T19:02:40.060953+0000","last_unstale":"2024-04-25T19:02:45.131214+0000","last_undegraded":"2024-04-25T19:02:45.131214+0000","last_fullsized":"2024-04-25T19:02:45.131214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132354+0000","last_change":"2024-04-25T19:02:39.044906+0000","last_active":"2024-04-25T19:02:45.132354+0000","last_peered":"2024-04-25T19:02:45.132354+0000","last_clean":"2024-04-25T19:02:45.132354+0000","last_became_active":"2024-04-25T19:02:39.043065+0000","last_became_peered":"2024-04-25T19:02:39.043065+0000","last_unstale":"2024-04-25T19:02:45.132354+0000","last_undegraded":"2024-04-25T19:02:45.132354+0000","last_fullsized":"2024-04-25T19:02:45.132354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203780+0000","last_change":"2024-04-25T19:02:44.149122+0000","last_active":"2024-04-25T19:02:45.203780+0000","last_peered":"2024-04-25T19:02:45.203780+0000","last_clean":"2024-04-25T19:02:45.203780+0000","last_became_active":"2024-04-25T19:02:44.148633+0000","last_became_peered":"2024-04-25T19:02:44.148633+0000","last_unstale":"2024-04-25T19:02:45.203780+0000","last_undegraded":"2024-04-25T19:02:45.203780+0000","last_fullsized":"2024-04-25T19:02:45.203780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128923+0000","last_change":"2024-04-25T19:02:40.067343+0000","last_active":"2024-04-25T19:02:45.128923+0000","last_peered":"2024-04-25T19:02:45.128923+0000","last_clean":"2024-04-25T19:02:45.128923+0000","last_became_active":"2024-04-25T19:02:40.066748+0000","last_became_peered":"2024-04-25T19:02:40.066748+0000","last_unstale":"2024-04-25T19:02:45.128923+0000","last_undegraded":"2024-04-25T19:02:45.128923+0000","last_fullsized":"2024-04-25T19:02:45.128923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:18.495508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196634+0000","last_change":"2024-04-25T19:02:39.042622+0000","last_active":"2024-04-25T19:02:45.196634+0000","last_peered":"2024-04-25T19:02:45.196634+0000","last_clean":"2024-04-25T19:02:45.196634+0000","last_became_active":"2024-04-25T19:02:39.042302+0000","last_became_peered":"2024-04-25T19:02:39.042302+0000","last_unstale":"2024-04-25T19:02:45.196634+0000","last_undegraded":"2024-04-25T19:02:45.196634+0000","last_fullsized":"2024-04-25T19:02:45.196634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199162+0000","last_change":"2024-04-25T19:02:44.148027+0000","last_active":"2024-04-25T19:02:45.199162+0000","last_peered":"2024-04-25T19:02:45.199162+0000","last_clean":"2024-04-25T19:02:45.199162+0000","last_became_active":"2024-04-25T19:02:44.147612+0000","last_became_peered":"2024-04-25T19:02:44.147612+0000","last_unstale":"2024-04-25T19:02:45.199162+0000","last_undegraded":"2024-04-25T19:02:45.199162+0000","last_fullsized":"2024-04-25T19:02:45.199162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:52.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136112+0000","last_change":"2024-04-25T19:02:40.062872+0000","last_active":"2024-04-25T19:02:45.136112+0000","last_peered":"2024-04-25T19:02:45.136112+0000","last_clean":"2024-04-25T19:02:45.136112+0000","last_became_active":"2024-04-25T19:02:40.061519+0000","last_became_peered":"2024-04-25T19:02:40.061519+0000","last_unstale":"2024-04-25T19:02:45.136112+0000","last_undegraded":"2024-04-25T19:02:45.136112+0000","last_fullsized":"2024-04-25T19:02:45.136112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132193+0000","last_change":"2024-04-25T19:02:39.054460+0000","last_active":"2024-04-25T19:02:45.132193+0000","last_peered":"2024-04-25T19:02:45.132193+0000","last_clean":"2024-04-25T19:02:45.132193+0000","last_became_active":"2024-04-25T19:02:39.053233+0000","last_became_peered":"2024-04-25T19:02:39.053233+0000","last_unstale":"2024-04-25T19:02:45.132193+0000","last_undegraded":"2024-04-25T19:02:45.132193+0000","last_fullsized":"2024-04-25T19:02:45.132193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:17.622604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199607+0000","last_change":"2024-04-25T19:02:44.139268+0000","last_active":"2024-04-25T19:02:45.199607+0000","last_peered":"2024-04-25T19:02:45.199607+0000","last_clean":"2024-04-25T19:02:45.199607+0000","last_became_active":"2024-04-25T19:02:44.137641+0000","last_became_peered":"2024-04-25T19:02:44.137641+0000","last_unstale":"2024-04-25T19:02:45.199607+0000","last_undegraded":"2024-04-25T19:02:45.199607+0000","last_fullsized":"2024-04-25T19:02:45.199607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137364+0000","last_change":"2024-04-25T19:02:40.063483+0000","last_active":"2024-04-25T19:02:45.137364+0000","last_peered":"2024-04-25T19:02:45.137364+0000","last_clean":"2024-04-25T19:02:45.137364+0000","last_became_active":"2024-04-25T19:02:40.062531+0000","last_became_peered":"2024-04-25T19:02:40.062531+0000","last_unstale":"2024-04-25T19:02:45.137364+0000","last_undegraded":"2024-04-25T19:02:45.137364+0000","last_fullsized":"2024-04-25T19:02:45.137364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136358+0000","last_change":"2024-04-25T19:02:39.052714+0000","last_active":"2024-04-25T19:02:45.136358+0000","last_peered":"2024-04-25T19:02:45.136358+0000","last_clean":"2024-04-25T19:02:45.136358+0000","last_became_active":"2024-04-25T19:02:39.051543+0000","last_became_peered":"2024-04-25T19:02:39.051543+0000","last_unstale":"2024-04-25T19:02:45.136358+0000","last_undegraded":"2024-04-25T19:02:45.136358+0000","last_fullsized":"2024-04-25T19:02:45.136358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131504+0000","last_change":"2024-04-25T19:02:44.142273+0000","last_active":"2024-04-25T19:02:45.131504+0000","last_peered":"2024-04-25T19:02:45.131504+0000","last_clean":"2024-04-25T19:02:45.131504+0000","last_became_active":"2024-04-25T19:02:44.140876+0000","last_became_peered":"2024-04-25T19:02:44.140876+0000","last_unstale":"2024-04-25T19:02:45.131504+0000","last_undegraded":"2024-04-25T19:02:45.131504+0000","last_fullsized":"2024-04-25T19:02:45.131504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137451+0000","last_change":"2024-04-25T19:02:40.065602+0000","last_active":"2024-04-25T19:02:45.137451+0000","last_peered":"2024-04-25T19:02:45.137451+0000","last_clean":"2024-04-25T19:02:45.137451+0000","last_became_active":"2024-04-25T19:02:40.065178+0000","last_became_peered":"2024-04-25T19:02:40.065178+0000","last_unstale":"2024-04-25T19:02:45.137451+0000","last_undegraded":"2024-04-25T19:02:45.137451+0000","last_fullsized":"2024-04-25T19:02:45.137451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.101532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131078+0000","last_change":"2024-04-25T19:02:39.054975+0000","last_active":"2024-04-25T19:02:45.131078+0000","last_peered":"2024-04-25T19:02:45.131078+0000","last_clean":"2024-04-25T19:02:45.131078+0000","last_became_active":"2024-04-25T19:02:39.052591+0000","last_became_peered":"2024-04-25T19:02:39.052591+0000","last_unstale":"2024-04-25T19:02:45.131078+0000","last_undegraded":"2024-04-25T19:02:45.131078+0000","last_fullsized":"2024-04-25T19:02:45.131078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:10.725639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135617+0000","last_change":"2024-04-25T19:02:44.144565+0000","last_active":"2024-04-25T19:02:45.135617+0000","last_peered":"2024-04-25T19:02:45.135617+0000","last_clean":"2024-04-25T19:02:45.135617+0000","last_became_active":"2024-04-25T19:02:44.144092+0000","last_became_peered":"2024-04-25T19:02:44.144092+0000","last_unstale":"2024-04-25T19:02:45.135617+0000","last_undegraded":"2024-04-25T19:02:45.135617+0000","last_fullsized":"2024-04-25T19:02:45.135617+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:18.177897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198835+0000","last_change":"2024-04-25T19:02:40.062748+0000","last_active":"2024-04-25T19:02:45.198835+0000","last_peered":"2024-04-25T19:02:45.198835+0000","last_clean":"2024-04-25T19:02:45.198835+0000","last_became_active":"2024-04-25T19:02:40.060618+0000","last_became_peered":"2024-04-25T19:02:40.060618+0000","last_unstale":"2024-04-25T19:02:45.198835+0000","last_undegraded":"2024-04-25T19:02:45.198835+0000","last_fullsized":"2024-04-25T19:02:45.198835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200394+0000","last_change":"2024-04-25T19:02:39.043998+0000","last_active":"2024-04-25T19:02:45.200394+0000","last_peered":"2024-04-25T19:02:45.200394+0000","last_clean":"2024-04-25T19:02:45.200394+0000","last_became_active":"2024-04-25T19:02:39.043623+0000","last_became_peered":"2024-04-25T19:02:39.043623+0000","last_unstale":"2024-04-25T19:02:45.200394+0000","last_undegraded":"2024-04-25T19:02:45.200394+0000","last_fullsized":"2024-04-25T19:02:45.200394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133644+0000","last_change":"2024-04-25T19:02:44.158919+0000","last_active":"2024-04-25T19:02:45.133644+0000","last_peered":"2024-04-25T19:02:45.133644+0000","last_clean":"2024-04-25T19:02:45.133644+0000","last_became_active":"2024-04-25T19:02:44.158358+0000","last_became_peered":"2024-04-25T19:02:44.158358+0000","last_unstale":"2024-04-25T19:02:45.133644+0000","last_undegraded":"2024-04-25T19:02:45.133644+0000","last_fullsized":"2024-04-25T19:02:45.133644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:05.831945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198482+0000","last_change":"2024-04-25T19:02:40.072547+0000","last_active":"2024-04-25T19:02:45.198482+0000","last_peered":"2024-04-25T19:02:45.198482+0000","last_clean":"2024-04-25T19:02:45.198482+0000","last_became_active":"2024-04-25T19:02:40.072124+0000","last_became_peered":"2024-04-25T19:02:40.072124+0000","last_unstale":"2024-04-25T19:02:45.198482+0000","last_undegraded":"2024-04-25T19:02:45.198482+0000","last_fullsized":"2024-04-25T19:02:45.198482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130699+0000","last_change":"2024-04-25T19:02:39.051040+0000","last_active":"2024-04-25T19:02:45.130699+0000","last_peered":"2024-04-25T19:02:45.130699+0000","last_clean":"2024-04-25T19:02:45.130699+0000","last_became_active":"2024-04-25T19:02:39.048647+0000","last_became_peered":"2024-04-25T19:02:39.048647+0000","last_unstale":"2024-04-25T19:02:45.130699+0000","last_undegraded":"2024-04-25T19:02:45.130699+0000","last_fullsized":"2024-04-25T19:02:45.130699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:53.187342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136962+0000","last_change":"2024-04-25T19:02:44.151392+0000","last_active":"2024-04-25T19:02:45.136962+0000","last_peered":"2024-04-25T19:02:45.136962+0000","last_clean":"2024-04-25T19:02:45.136962+0000","last_became_active":"2024-04-25T19:02:44.150074+0000","last_became_peered":"2024-04-25T19:02:44.150074+0000","last_unstale":"2024-04-25T19:02:45.136962+0000","last_undegraded":"2024-04-25T19:02:45.136962+0000","last_fullsized":"2024-04-25T19:02:45.136962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:03.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199109+0000","last_change":"2024-04-25T19:02:40.074602+0000","last_active":"2024-04-25T19:02:45.199109+0000","last_peered":"2024-04-25T19:02:45.199109+0000","last_clean":"2024-04-25T19:02:45.199109+0000","last_became_active":"2024-04-25T19:02:40.070530+0000","last_became_peered":"2024-04-25T19:02:40.070530+0000","last_unstale":"2024-04-25T19:02:45.199109+0000","last_undegraded":"2024-04-25T19:02:45.199109+0000","last_fullsized":"2024-04-25T19:02:45.199109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200966+0000","last_change":"2024-04-25T19:02:39.048859+0000","last_active":"2024-04-25T19:02:45.200966+0000","last_peered":"2024-04-25T19:02:45.200966+0000","last_clean":"2024-04-25T19:02:45.200966+0000","last_became_active":"2024-04-25T19:02:39.048497+0000","last_became_peered":"2024-04-25T19:02:39.048497+0000","last_unstale":"2024-04-25T19:02:45.200966+0000","last_undegraded":"2024-04-25T19:02:45.200966+0000","last_fullsized":"2024-04-25T19:02:45.200966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:13.519715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137567+0000","last_change":"2024-04-25T19:02:44.150754+0000","last_active":"2024-04-25T19:02:45.137567+0000","last_peered":"2024-04-25T19:02:45.137567+0000","last_clean":"2024-04-25T19:02:45.137567+0000","last_became_active":"2024-04-25T19:02:44.150092+0000","last_became_peered":"2024-04-25T19:02:44.150092+0000","last_unstale":"2024-04-25T19:02:45.137567+0000","last_undegraded":"2024-04-25T19:02:45.137567+0000","last_fullsized":"2024-04-25T19:02:45.137567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:45.602374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132688+0000","last_change":"2024-04-25T19:02:40.068029+0000","last_active":"2024-04-25T19:02:45.132688+0000","last_peered":"2024-04-25T19:02:45.132688+0000","last_clean":"2024-04-25T19:02:45.132688+0000","last_became_active":"2024-04-25T19:02:40.066355+0000","last_became_peered":"2024-04-25T19:02:40.066355+0000","last_unstale":"2024-04-25T19:02:45.132688+0000","last_undegraded":"2024-04-25T19:02:45.132688+0000","last_fullsized":"2024-04-25T19:02:45.132688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:09.936302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129192+0000","last_change":"2024-04-25T19:02:39.047056+0000","last_active":"2024-04-25T19:02:45.129192+0000","last_peered":"2024-04-25T19:02:45.129192+0000","last_clean":"2024-04-25T19:02:45.129192+0000","last_became_active":"2024-04-25T19:02:39.046198+0000","last_became_peered":"2024-04-25T19:02:39.046198+0000","last_unstale":"2024-04-25T19:02:45.129192+0000","last_undegraded":"2024-04-25T19:02:45.129192+0000","last_fullsized":"2024-04-25T19:02:45.129192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202550+0000","last_change":"2024-04-25T19:02:44.158508+0000","last_active":"2024-04-25T19:02:45.202550+0000","last_peered":"2024-04-25T19:02:45.202550+0000","last_clean":"2024-04-25T19:02:45.202550+0000","last_became_active":"2024-04-25T19:02:44.157734+0000","last_became_peered":"2024-04-25T19:02:44.157734+0000","last_unstale":"2024-04-25T19:02:45.202550+0000","last_undegraded":"2024-04-25T19:02:45.202550+0000","last_fullsized":"2024-04-25T19:02:45.202550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:42.894635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133801+0000","last_change":"2024-04-25T19:02:40.048821+0000","last_active":"2024-04-25T19:02:45.133801+0000","last_peered":"2024-04-25T19:02:45.133801+0000","last_clean":"2024-04-25T19:02:45.133801+0000","last_became_active":"2024-04-25T19:02:40.048295+0000","last_became_peered":"2024-04-25T19:02:40.048295+0000","last_unstale":"2024-04-25T19:02:45.133801+0000","last_undegraded":"2024-04-25T19:02:45.133801+0000","last_fullsized":"2024-04-25T19:02:45.133801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129911+0000","last_change":"2024-04-25T19:02:39.036175+0000","last_active":"2024-04-25T19:02:45.129911+0000","last_peered":"2024-04-25T19:02:45.129911+0000","last_clean":"2024-04-25T19:02:45.129911+0000","last_became_active":"2024-04-25T19:02:39.035822+0000","last_became_peered":"2024-04-25T19:02:39.035822+0000","last_unstale":"2024-04-25T19:02:45.129911+0000","last_undegraded":"2024-04-25T19:02:45.129911+0000","last_fullsized":"2024-04-25T19:02:45.129911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.306938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197039+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:02:45.197039+0000","last_peered":"2024-04-25T19:02:45.197039+0000","last_clean":"2024-04-25T19:02:45.197039+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:02:45.197039+0000","last_undegraded":"2024-04-25T19:02:45.197039+0000","last_fullsized":"2024-04-25T19:02:45.197039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131234+0000","last_change":"2024-04-25T19:02:44.144534+0000","last_active":"2024-04-25T19:02:45.131234+0000","last_peered":"2024-04-25T19:02:45.131234+0000","last_clean":"2024-04-25T19:02:45.131234+0000","last_became_active":"2024-04-25T19:02:44.143365+0000","last_became_peered":"2024-04-25T19:02:44.143365+0000","last_unstale":"2024-04-25T19:02:45.131234+0000","last_undegraded":"2024-04-25T19:02:45.131234+0000","last_fullsized":"2024-04-25T19:02:45.131234+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:39.667305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130665+0000","last_change":"2024-04-25T19:02:40.082915+0000","last_active":"2024-04-25T19:02:45.130665+0000","last_peered":"2024-04-25T19:02:45.130665+0000","last_clean":"2024-04-25T19:02:45.130665+0000","last_became_active":"2024-04-25T19:02:40.082594+0000","last_became_peered":"2024-04-25T19:02:40.082594+0000","last_unstale":"2024-04-25T19:02:45.130665+0000","last_undegraded":"2024-04-25T19:02:45.130665+0000","last_fullsized":"2024-04-25T19:02:45.130665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:10.588191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137737+0000","last_change":"2024-04-25T19:02:39.052990+0000","last_active":"2024-04-25T19:02:45.137737+0000","last_peered":"2024-04-25T19:02:45.137737+0000","last_clean":"2024-04-25T19:02:45.137737+0000","last_became_active":"2024-04-25T19:02:39.052635+0000","last_became_peered":"2024-04-25T19:02:39.052635+0000","last_unstale":"2024-04-25T19:02:45.137737+0000","last_undegraded":"2024-04-25T19:02:45.137737+0000","last_fullsized":"2024-04-25T19:02:45.137737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.298378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198050+0000","last_change":"2024-04-25T19:02:44.143773+0000","last_active":"2024-04-25T19:02:45.198050+0000","last_peered":"2024-04-25T19:02:45.198050+0000","last_clean":"2024-04-25T19:02:45.198050+0000","last_became_active":"2024-04-25T19:02:44.141602+0000","last_became_peered":"2024-04-25T19:02:44.141602+0000","last_unstale":"2024-04-25T19:02:45.198050+0000","last_undegraded":"2024-04-25T19:02:45.198050+0000","last_fullsized":"2024-04-25T19:02:45.198050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:03.052112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199104+0000","last_change":"2024-04-25T19:02:40.075177+0000","last_active":"2024-04-25T19:02:45.199104+0000","last_peered":"2024-04-25T19:02:45.199104+0000","last_clean":"2024-04-25T19:02:45.199104+0000","last_became_active":"2024-04-25T19:02:40.074512+0000","last_became_peered":"2024-04-25T19:02:40.074512+0000","last_unstale":"2024-04-25T19:02:45.199104+0000","last_undegraded":"2024-04-25T19:02:45.199104+0000","last_fullsized":"2024-04-25T19:02:45.199104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201661+0000","last_change":"2024-04-25T19:02:39.052054+0000","last_active":"2024-04-25T19:02:45.201661+0000","last_peered":"2024-04-25T19:02:45.201661+0000","last_clean":"2024-04-25T19:02:45.201661+0000","last_became_active":"2024-04-25T19:02:39.051646+0000","last_became_peered":"2024-04-25T19:02:39.051646+0000","last_unstale":"2024-04-25T19:02:45.201661+0000","last_undegraded":"2024-04-25T19:02:45.201661+0000","last_fullsized":"2024-04-25T19:02:45.201661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200618+0000","last_change":"2024-04-25T19:02:44.157149+0000","last_active":"2024-04-25T19:02:45.200618+0000","last_peered":"2024-04-25T19:02:45.200618+0000","last_clean":"2024-04-25T19:02:45.200618+0000","last_became_active":"2024-04-25T19:02:44.156656+0000","last_became_peered":"2024-04-25T19:02:44.156656+0000","last_unstale":"2024-04-25T19:02:45.200618+0000","last_undegraded":"2024-04-25T19:02:45.200618+0000","last_fullsized":"2024-04-25T19:02:45.200618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:14.674405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198854+0000","last_change":"2024-04-25T19:02:40.080039+0000","last_active":"2024-04-25T19:02:45.198854+0000","last_peered":"2024-04-25T19:02:45.198854+0000","last_clean":"2024-04-25T19:02:45.198854+0000","last_became_active":"2024-04-25T19:02:40.079725+0000","last_became_peered":"2024-04-25T19:02:40.079725+0000","last_unstale":"2024-04-25T19:02:45.198854+0000","last_undegraded":"2024-04-25T19:02:45.198854+0000","last_fullsized":"2024-04-25T19:02:45.198854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136108+0000","last_change":"2024-04-25T19:02:39.051841+0000","last_active":"2024-04-25T19:02:45.136108+0000","last_peered":"2024-04-25T19:02:45.136108+0000","last_clean":"2024-04-25T19:02:45.136108+0000","last_became_active":"2024-04-25T19:02:39.050695+0000","last_became_peered":"2024-04-25T19:02:39.050695+0000","last_unstale":"2024-04-25T19:02:45.136108+0000","last_undegraded":"2024-04-25T19:02:45.136108+0000","last_fullsized":"2024-04-25T19:02:45.136108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128669+0000","last_change":"2024-04-25T19:02:44.147067+0000","last_active":"2024-04-25T19:02:45.128669+0000","last_peered":"2024-04-25T19:02:45.128669+0000","last_clean":"2024-04-25T19:02:45.128669+0000","last_became_active":"2024-04-25T19:02:44.146048+0000","last_became_peered":"2024-04-25T19:02:44.146048+0000","last_unstale":"2024-04-25T19:02:45.128669+0000","last_undegraded":"2024-04-25T19:02:45.128669+0000","last_fullsized":"2024-04-25T19:02:45.128669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:13.605566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199716+0000","last_change":"2024-04-25T19:02:40.066529+0000","last_active":"2024-04-25T19:02:45.199716+0000","last_peered":"2024-04-25T19:02:45.199716+0000","last_clean":"2024-04-25T19:02:45.199716+0000","last_became_active":"2024-04-25T19:02:40.064814+0000","last_became_peered":"2024-04-25T19:02:40.064814+0000","last_unstale":"2024-04-25T19:02:45.199716+0000","last_undegraded":"2024-04-25T19:02:45.199716+0000","last_fullsized":"2024-04-25T19:02:45.199716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199020+0000","last_change":"2024-04-25T19:02:39.051220+0000","last_active":"2024-04-25T19:02:45.199020+0000","last_peered":"2024-04-25T19:02:45.199020+0000","last_clean":"2024-04-25T19:02:45.199020+0000","last_became_active":"2024-04-25T19:02:39.050911+0000","last_became_peered":"2024-04-25T19:02:39.050911+0000","last_unstale":"2024-04-25T19:02:45.199020+0000","last_undegraded":"2024-04-25T19:02:45.199020+0000","last_fullsized":"2024-04-25T19:02:45.199020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196897+0000","last_change":"2024-04-25T19:02:44.156822+0000","last_active":"2024-04-25T19:02:45.196897+0000","last_peered":"2024-04-25T19:02:45.196897+0000","last_clean":"2024-04-25T19:02:45.196897+0000","last_became_active":"2024-04-25T19:02:44.156379+0000","last_became_peered":"2024-04-25T19:02:44.156379+0000","last_unstale":"2024-04-25T19:02:45.196897+0000","last_undegraded":"2024-04-25T19:02:45.196897+0000","last_fullsized":"2024-04-25T19:02:45.196897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128832+0000","last_change":"2024-04-25T19:02:40.075607+0000","last_active":"2024-04-25T19:02:45.128832+0000","last_peered":"2024-04-25T19:02:45.128832+0000","last_clean":"2024-04-25T19:02:45.128832+0000","last_became_active":"2024-04-25T19:02:40.075267+0000","last_became_peered":"2024-04-25T19:02:40.075267+0000","last_unstale":"2024-04-25T19:02:45.128832+0000","last_undegraded":"2024-04-25T19:02:45.128832+0000","last_fullsized":"2024-04-25T19:02:45.128832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131535+0000","last_change":"2024-04-25T19:02:39.044744+0000","last_active":"2024-04-25T19:02:45.131535+0000","last_peered":"2024-04-25T19:02:45.131535+0000","last_clean":"2024-04-25T19:02:45.131535+0000","last_became_active":"2024-04-25T19:02:39.044372+0000","last_became_peered":"2024-04-25T19:02:39.044372+0000","last_unstale":"2024-04-25T19:02:45.131535+0000","last_undegraded":"2024-04-25T19:02:45.131535+0000","last_fullsized":"2024-04-25T19:02:45.131535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203015+0000","last_change":"2024-04-25T19:02:44.158146+0000","last_active":"2024-04-25T19:02:45.203015+0000","last_peered":"2024-04-25T19:02:45.203015+0000","last_clean":"2024-04-25T19:02:45.203015+0000","last_became_active":"2024-04-25T19:02:44.157017+0000","last_became_peered":"2024-04-25T19:02:44.157017+0000","last_unstale":"2024-04-25T19:02:45.203015+0000","last_undegraded":"2024-04-25T19:02:45.203015+0000","last_fullsized":"2024-04-25T19:02:45.203015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:40.012791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196618+0000","last_change":"2024-04-25T19:02:40.073754+0000","last_active":"2024-04-25T19:02:45.196618+0000","last_peered":"2024-04-25T19:02:45.196618+0000","last_clean":"2024-04-25T19:02:45.196618+0000","last_became_active":"2024-04-25T19:02:40.073500+0000","last_became_peered":"2024-04-25T19:02:40.073500+0000","last_unstale":"2024-04-25T19:02:45.196618+0000","last_undegraded":"2024-04-25T19:02:45.196618+0000","last_fullsized":"2024-04-25T19:02:45.196618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197302+0000","last_change":"2024-04-25T19:02:39.039435+0000","last_active":"2024-04-25T19:02:45.197302+0000","last_peered":"2024-04-25T19:02:45.197302+0000","last_clean":"2024-04-25T19:02:45.197302+0000","last_became_active":"2024-04-25T19:02:39.039084+0000","last_became_peered":"2024-04-25T19:02:39.039084+0000","last_unstale":"2024-04-25T19:02:45.197302+0000","last_undegraded":"2024-04-25T19:02:45.197302+0000","last_fullsized":"2024-04-25T19:02:45.197302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202918+0000","last_change":"2024-04-25T19:02:44.156161+0000","last_active":"2024-04-25T19:02:45.202918+0000","last_peered":"2024-04-25T19:02:45.202918+0000","last_clean":"2024-04-25T19:02:45.202918+0000","last_became_active":"2024-04-25T19:02:44.155666+0000","last_became_peered":"2024-04-25T19:02:44.155666+0000","last_unstale":"2024-04-25T19:02:45.202918+0000","last_undegraded":"2024-04-25T19:02:45.202918+0000","last_fullsized":"2024-04-25T19:02:45.202918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:22:00.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129573+0000","last_change":"2024-04-25T19:02:40.076702+0000","last_active":"2024-04-25T19:02:45.129573+0000","last_peered":"2024-04-25T19:02:45.129573+0000","last_clean":"2024-04-25T19:02:45.129573+0000","last_became_active":"2024-04-25T19:02:40.072663+0000","last_became_peered":"2024-04-25T19:02:40.072663+0000","last_unstale":"2024-04-25T19:02:45.129573+0000","last_undegraded":"2024-04-25T19:02:45.129573+0000","last_fullsized":"2024-04-25T19:02:45.129573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:35.926577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130496+0000","last_change":"2024-04-25T19:02:39.048307+0000","last_active":"2024-04-25T19:02:45.130496+0000","last_peered":"2024-04-25T19:02:45.130496+0000","last_clean":"2024-04-25T19:02:45.130496+0000","last_became_active":"2024-04-25T19:02:39.047902+0000","last_became_peered":"2024-04-25T19:02:39.047902+0000","last_unstale":"2024-04-25T19:02:45.130496+0000","last_undegraded":"2024-04-25T19:02:45.130496+0000","last_fullsized":"2024-04-25T19:02:45.130496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136907+0000","last_change":"2024-04-25T19:02:44.151849+0000","last_active":"2024-04-25T19:02:45.136907+0000","last_peered":"2024-04-25T19:02:45.136907+0000","last_clean":"2024-04-25T19:02:45.136907+0000","last_became_active":"2024-04-25T19:02:44.150781+0000","last_became_peered":"2024-04-25T19:02:44.150781+0000","last_unstale":"2024-04-25T19:02:45.136907+0000","last_undegraded":"2024-04-25T19:02:45.136907+0000","last_fullsized":"2024-04-25T19:02:45.136907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:30.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134288+0000","last_change":"2024-04-25T19:02:40.073292+0000","last_active":"2024-04-25T19:02:45.134288+0000","last_peered":"2024-04-25T19:02:45.134288+0000","last_clean":"2024-04-25T19:02:45.134288+0000","last_became_active":"2024-04-25T19:02:40.070890+0000","last_became_peered":"2024-04-25T19:02:40.070890+0000","last_unstale":"2024-04-25T19:02:45.134288+0000","last_undegraded":"2024-04-25T19:02:45.134288+0000","last_fullsized":"2024-04-25T19:02:45.134288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:19.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201740+0000","last_change":"2024-04-25T19:02:39.053287+0000","last_active":"2024-04-25T19:02:45.201740+0000","last_peered":"2024-04-25T19:02:45.201740+0000","last_clean":"2024-04-25T19:02:45.201740+0000","last_became_active":"2024-04-25T19:02:39.052037+0000","last_became_peered":"2024-04-25T19:02:39.052037+0000","last_unstale":"2024-04-25T19:02:45.201740+0000","last_undegraded":"2024-04-25T19:02:45.201740+0000","last_fullsized":"2024-04-25T19:02:45.201740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130560+0000","last_change":"2024-04-25T19:02:44.158840+0000","last_active":"2024-04-25T19:02:45.130560+0000","last_peered":"2024-04-25T19:02:45.130560+0000","last_clean":"2024-04-25T19:02:45.130560+0000","last_became_active":"2024-04-25T19:02:44.156508+0000","last_became_peered":"2024-04-25T19:02:44.156508+0000","last_unstale":"2024-04-25T19:02:45.130560+0000","last_undegraded":"2024-04-25T19:02:45.130560+0000","last_fullsized":"2024-04-25T19:02:45.130560+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:08.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198990+0000","last_change":"2024-04-25T19:02:40.084277+0000","last_active":"2024-04-25T19:02:45.198990+0000","last_peered":"2024-04-25T19:02:45.198990+0000","last_clean":"2024-04-25T19:02:45.198990+0000","last_became_active":"2024-04-25T19:02:40.083896+0000","last_became_peered":"2024-04-25T19:02:40.083896+0000","last_unstale":"2024-04-25T19:02:45.198990+0000","last_undegraded":"2024-04-25T19:02:45.198990+0000","last_fullsized":"2024-04-25T19:02:45.198990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.393874+0000","last_change":"2024-04-25T19:02:39.045230+0000","last_active":"2024-04-25T19:03:00.393874+0000","last_peered":"2024-04-25T19:03:00.393874+0000","last_clean":"2024-04-25T19:03:00.393874+0000","last_became_active":"2024-04-25T19:02:39.044884+0000","last_became_peered":"2024-04-25T19:02:39.044884+0000","last_unstale":"2024-04-25T19:03:00.393874+0000","last_undegraded":"2024-04-25T19:03:00.393874+0000","last_fullsized":"2024-04-25T19:03:00.393874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:20.905809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133971+0000","last_change":"2024-04-25T19:02:44.153433+0000","last_active":"2024-04-25T19:02:45.133971+0000","last_peered":"2024-04-25T19:02:45.133971+0000","last_clean":"2024-04-25T19:02:45.133971+0000","last_became_active":"2024-04-25T19:02:44.152935+0000","last_became_peered":"2024-04-25T19:02:44.152935+0000","last_unstale":"2024-04-25T19:02:45.133971+0000","last_undegraded":"2024-04-25T19:02:45.133971+0000","last_fullsized":"2024-04-25T19:02:45.133971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:35.591324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129614+0000","last_change":"2024-04-25T19:02:40.074340+0000","last_active":"2024-04-25T19:02:45.129614+0000","last_peered":"2024-04-25T19:02:45.129614+0000","last_clean":"2024-04-25T19:02:45.129614+0000","last_became_active":"2024-04-25T19:02:40.069150+0000","last_became_peered":"2024-04-25T19:02:40.069150+0000","last_unstale":"2024-04-25T19:02:45.129614+0000","last_undegraded":"2024-04-25T19:02:45.129614+0000","last_fullsized":"2024-04-25T19:02:45.129614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:53.328375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203162+0000","last_change":"2024-04-25T19:02:39.050352+0000","last_active":"2024-04-25T19:02:45.203162+0000","last_peered":"2024-04-25T19:02:45.203162+0000","last_clean":"2024-04-25T19:02:45.203162+0000","last_became_active":"2024-04-25T19:02:39.049837+0000","last_became_peered":"2024-04-25T19:02:39.049837+0000","last_unstale":"2024-04-25T19:02:45.203162+0000","last_undegraded":"2024-04-25T19:02:45.203162+0000","last_fullsized":"2024-04-25T19:02:45.203162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:46.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200045+0000","last_change":"2024-04-25T19:02:44.147062+0000","last_active":"2024-04-25T19:02:45.200045+0000","last_peered":"2024-04-25T19:02:45.200045+0000","last_clean":"2024-04-25T19:02:45.200045+0000","last_became_active":"2024-04-25T19:02:44.145997+0000","last_became_peered":"2024-04-25T19:02:44.145997+0000","last_unstale":"2024-04-25T19:02:45.200045+0000","last_undegraded":"2024-04-25T19:02:45.200045+0000","last_fullsized":"2024-04-25T19:02:45.200045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203397+0000","last_change":"2024-04-25T19:02:40.083756+0000","last_active":"2024-04-25T19:02:45.203397+0000","last_peered":"2024-04-25T19:02:45.203397+0000","last_clean":"2024-04-25T19:02:45.203397+0000","last_became_active":"2024-04-25T19:02:40.083402+0000","last_became_peered":"2024-04-25T19:02:40.083402+0000","last_unstale":"2024-04-25T19:02:45.203397+0000","last_undegraded":"2024-04-25T19:02:45.203397+0000","last_fullsized":"2024-04-25T19:02:45.203397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198142+0000","last_change":"2024-04-25T19:02:39.043121+0000","last_active":"2024-04-25T19:02:45.198142+0000","last_peered":"2024-04-25T19:02:45.198142+0000","last_clean":"2024-04-25T19:02:45.198142+0000","last_became_active":"2024-04-25T19:02:39.042778+0000","last_became_peered":"2024-04-25T19:02:39.042778+0000","last_unstale":"2024-04-25T19:02:45.198142+0000","last_undegraded":"2024-04-25T19:02:45.198142+0000","last_fullsized":"2024-04-25T19:02:45.198142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199456+0000","last_change":"2024-04-25T19:02:44.148628+0000","last_active":"2024-04-25T19:02:45.199456+0000","last_peered":"2024-04-25T19:02:45.199456+0000","last_clean":"2024-04-25T19:02:45.199456+0000","last_became_active":"2024-04-25T19:02:44.147929+0000","last_became_peered":"2024-04-25T19:02:44.147929+0000","last_unstale":"2024-04-25T19:02:45.199456+0000","last_undegraded":"2024-04-25T19:02:45.199456+0000","last_fullsized":"2024-04-25T19:02:45.199456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199371+0000","last_change":"2024-04-25T19:02:40.069980+0000","last_active":"2024-04-25T19:02:45.199371+0000","last_peered":"2024-04-25T19:02:45.199371+0000","last_clean":"2024-04-25T19:02:45.199371+0000","last_became_active":"2024-04-25T19:02:40.069323+0000","last_became_peered":"2024-04-25T19:02:40.069323+0000","last_unstale":"2024-04-25T19:02:45.199371+0000","last_undegraded":"2024-04-25T19:02:45.199371+0000","last_fullsized":"2024-04-25T19:02:45.199371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200662+0000","last_change":"2024-04-25T19:02:39.044027+0000","last_active":"2024-04-25T19:02:45.200662+0000","last_peered":"2024-04-25T19:02:45.200662+0000","last_clean":"2024-04-25T19:02:45.200662+0000","last_became_active":"2024-04-25T19:02:39.043696+0000","last_became_peered":"2024-04-25T19:02:39.043696+0000","last_unstale":"2024-04-25T19:02:45.200662+0000","last_undegraded":"2024-04-25T19:02:45.200662+0000","last_fullsized":"2024-04-25T19:02:45.200662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132192+0000","last_change":"2024-04-25T19:02:44.147513+0000","last_active":"2024-04-25T19:02:45.132192+0000","last_peered":"2024-04-25T19:02:45.132192+0000","last_clean":"2024-04-25T19:02:45.132192+0000","last_became_active":"2024-04-25T19:02:44.146937+0000","last_became_peered":"2024-04-25T19:02:44.146937+0000","last_unstale":"2024-04-25T19:02:45.132192+0000","last_undegraded":"2024-04-25T19:02:45.132192+0000","last_fullsized":"2024-04-25T19:02:45.132192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135996+0000","last_change":"2024-04-25T19:02:44.153340+0000","last_active":"2024-04-25T19:02:45.135996+0000","last_peered":"2024-04-25T19:02:45.135996+0000","last_clean":"2024-04-25T19:02:45.135996+0000","last_became_active":"2024-04-25T19:02:44.152850+0000","last_became_peered":"2024-04-25T19:02:44.152850+0000","last_unstale":"2024-04-25T19:02:45.135996+0000","last_undegraded":"2024-04-25T19:02:45.135996+0000","last_fullsized":"2024-04-25T19:02:45.135996+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:57.414588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132085+0000","last_change":"2024-04-25T19:02:40.051844+0000","last_active":"2024-04-25T19:02:45.132085+0000","last_peered":"2024-04-25T19:02:45.132085+0000","last_clean":"2024-04-25T19:02:45.132085+0000","last_became_active":"2024-04-25T19:02:40.050922+0000","last_became_peered":"2024-04-25T19:02:40.050922+0000","last_unstale":"2024-04-25T19:02:45.132085+0000","last_undegraded":"2024-04-25T19:02:45.132085+0000","last_fullsized":"2024-04-25T19:02:45.132085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.344370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129179+0000","last_change":"2024-04-25T19:02:39.045270+0000","last_active":"2024-04-25T19:02:45.129179+0000","last_peered":"2024-04-25T19:02:45.129179+0000","last_clean":"2024-04-25T19:02:45.129179+0000","last_became_active":"2024-04-25T19:02:39.044936+0000","last_became_peered":"2024-04-25T19:02:39.044936+0000","last_unstale":"2024-04-25T19:02:45.129179+0000","last_undegraded":"2024-04-25T19:02:45.129179+0000","last_fullsized":"2024-04-25T19:02:45.129179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131178+0000","last_change":"2024-04-25T19:02:44.160881+0000","last_active":"2024-04-25T19:02:45.131178+0000","last_peered":"2024-04-25T19:02:45.131178+0000","last_clean":"2024-04-25T19:02:45.131178+0000","last_became_active":"2024-04-25T19:02:44.160267+0000","last_became_peered":"2024-04-25T19:02:44.160267+0000","last_unstale":"2024-04-25T19:02:45.131178+0000","last_undegraded":"2024-04-25T19:02:45.131178+0000","last_fullsized":"2024-04-25T19:02:45.131178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:21.942974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199361+0000","last_change":"2024-04-25T19:02:40.065898+0000","last_active":"2024-04-25T19:02:45.199361+0000","last_peered":"2024-04-25T19:02:45.199361+0000","last_clean":"2024-04-25T19:02:45.199361+0000","last_became_active":"2024-04-25T19:02:40.062397+0000","last_became_peered":"2024-04-25T19:02:40.062397+0000","last_unstale":"2024-04-25T19:02:45.199361+0000","last_undegraded":"2024-04-25T19:02:45.199361+0000","last_fullsized":"2024-04-25T19:02:45.199361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132878+0000","last_change":"2024-04-25T19:02:39.049593+0000","last_active":"2024-04-25T19:02:45.132878+0000","last_peered":"2024-04-25T19:02:45.132878+0000","last_clean":"2024-04-25T19:02:45.132878+0000","last_became_active":"2024-04-25T19:02:39.049140+0000","last_became_peered":"2024-04-25T19:02:39.049140+0000","last_unstale":"2024-04-25T19:02:45.132878+0000","last_undegraded":"2024-04-25T19:02:45.132878+0000","last_fullsized":"2024-04-25T19:02:45.132878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:01.624014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130188+0000","last_change":"2024-04-25T19:02:39.050611+0000","last_active":"2024-04-25T19:02:45.130188+0000","last_peered":"2024-04-25T19:02:45.130188+0000","last_clean":"2024-04-25T19:02:45.130188+0000","last_became_active":"2024-04-25T19:02:39.050319+0000","last_became_peered":"2024-04-25T19:02:39.050319+0000","last_unstale":"2024-04-25T19:02:45.130188+0000","last_undegraded":"2024-04-25T19:02:45.130188+0000","last_fullsized":"2024-04-25T19:02:45.130188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132040+0000","last_change":"2024-04-25T19:02:40.076010+0000","last_active":"2024-04-25T19:02:45.132040+0000","last_peered":"2024-04-25T19:02:45.132040+0000","last_clean":"2024-04-25T19:02:45.132040+0000","last_became_active":"2024-04-25T19:02:40.075677+0000","last_became_peered":"2024-04-25T19:02:40.075677+0000","last_unstale":"2024-04-25T19:02:45.132040+0000","last_undegraded":"2024-04-25T19:02:45.132040+0000","last_fullsized":"2024-04-25T19:02:45.132040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135539+0000","last_change":"2024-04-25T19:02:44.154362+0000","last_active":"2024-04-25T19:02:45.135539+0000","last_peered":"2024-04-25T19:02:45.135539+0000","last_clean":"2024-04-25T19:02:45.135539+0000","last_became_active":"2024-04-25T19:02:44.153509+0000","last_became_peered":"2024-04-25T19:02:44.153509+0000","last_unstale":"2024-04-25T19:02:45.135539+0000","last_undegraded":"2024-04-25T19:02:45.135539+0000","last_fullsized":"2024-04-25T19:02:45.135539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:16.467033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201054+0000","last_change":"2024-04-25T19:02:39.045741+0000","last_active":"2024-04-25T19:02:45.201054+0000","last_peered":"2024-04-25T19:02:45.201054+0000","last_clean":"2024-04-25T19:02:45.201054+0000","last_became_active":"2024-04-25T19:02:39.045015+0000","last_became_peered":"2024-04-25T19:02:39.045015+0000","last_unstale":"2024-04-25T19:02:45.201054+0000","last_undegraded":"2024-04-25T19:02:45.201054+0000","last_fullsized":"2024-04-25T19:02:45.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199483+0000","last_change":"2024-04-25T19:02:40.080804+0000","last_active":"2024-04-25T19:02:45.199483+0000","last_peered":"2024-04-25T19:02:45.199483+0000","last_clean":"2024-04-25T19:02:45.199483+0000","last_became_active":"2024-04-25T19:02:40.080421+0000","last_became_peered":"2024-04-25T19:02:40.080421+0000","last_unstale":"2024-04-25T19:02:45.199483+0000","last_undegraded":"2024-04-25T19:02:45.199483+0000","last_fullsized":"2024-04-25T19:02:45.199483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202500+0000","last_change":"2024-04-25T19:02:44.163067+0000","last_active":"2024-04-25T19:02:45.202500+0000","last_peered":"2024-04-25T19:02:45.202500+0000","last_clean":"2024-04-25T19:02:45.202500+0000","last_became_active":"2024-04-25T19:02:44.162000+0000","last_became_peered":"2024-04-25T19:02:44.162000+0000","last_unstale":"2024-04-25T19:02:45.202500+0000","last_undegraded":"2024-04-25T19:02:45.202500+0000","last_fullsized":"2024-04-25T19:02:45.202500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:59.430583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198599+0000","last_change":"2024-04-25T19:02:40.063088+0000","last_active":"2024-04-25T19:02:45.198599+0000","last_peered":"2024-04-25T19:02:45.198599+0000","last_clean":"2024-04-25T19:02:45.198599+0000","last_became_active":"2024-04-25T19:02:40.062379+0000","last_became_peered":"2024-04-25T19:02:40.062379+0000","last_unstale":"2024-04-25T19:02:45.198599+0000","last_undegraded":"2024-04-25T19:02:45.198599+0000","last_fullsized":"2024-04-25T19:02:45.198599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200902+0000","last_change":"2024-04-25T19:02:40.065240+0000","last_active":"2024-04-25T19:02:45.200902+0000","last_peered":"2024-04-25T19:02:45.200902+0000","last_clean":"2024-04-25T19:02:45.200902+0000","last_became_active":"2024-04-25T19:02:40.064900+0000","last_became_peered":"2024-04-25T19:02:40.064900+0000","last_unstale":"2024-04-25T19:02:45.200902+0000","last_undegraded":"2024-04-25T19:02:45.200902+0000","last_fullsized":"2024-04-25T19:02:45.200902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:34.156431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129604+0000","last_change":"2024-04-25T19:02:40.084316+0000","last_active":"2024-04-25T19:02:45.129604+0000","last_peered":"2024-04-25T19:02:45.129604+0000","last_clean":"2024-04-25T19:02:45.129604+0000","last_became_active":"2024-04-25T19:02:40.084014+0000","last_became_peered":"2024-04-25T19:02:40.084014+0000","last_unstale":"2024-04-25T19:02:45.129604+0000","last_undegraded":"2024-04-25T19:02:45.129604+0000","last_fullsized":"2024-04-25T19:02:45.129604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200778+0000","last_change":"2024-04-25T19:02:40.070357+0000","last_active":"2024-04-25T19:02:45.200778+0000","last_peered":"2024-04-25T19:02:45.200778+0000","last_clean":"2024-04-25T19:02:45.200778+0000","last_became_active":"2024-04-25T19:02:40.070028+0000","last_became_peered":"2024-04-25T19:02:40.070028+0000","last_unstale":"2024-04-25T19:02:45.200778+0000","last_undegraded":"2024-04-25T19:02:45.200778+0000","last_fullsized":"2024-04-25T19:02:45.200778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137153+0000","last_change":"2024-04-25T19:02:40.088098+0000","last_active":"2024-04-25T19:02:45.137153+0000","last_peered":"2024-04-25T19:02:45.137153+0000","last_clean":"2024-04-25T19:02:45.137153+0000","last_became_active":"2024-04-25T19:02:40.087738+0000","last_became_peered":"2024-04-25T19:02:40.087738+0000","last_unstale":"2024-04-25T19:02:45.137153+0000","last_undegraded":"2024-04-25T19:02:45.137153+0000","last_fullsized":"2024-04-25T19:02:45.137153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:08.428103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137734+0000","last_change":"2024-04-25T19:02:40.063201+0000","last_active":"2024-04-25T19:02:45.137734+0000","last_peered":"2024-04-25T19:02:45.137734+0000","last_clean":"2024-04-25T19:02:45.137734+0000","last_became_active":"2024-04-25T19:02:40.060739+0000","last_became_peered":"2024-04-25T19:02:40.060739+0000","last_unstale":"2024-04-25T19:02:45.137734+0000","last_undegraded":"2024-04-25T19:02:45.137734+0000","last_fullsized":"2024-04-25T19:02:45.137734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:40.756295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199236+0000","last_change":"2024-04-25T19:02:40.076099+0000","last_active":"2024-04-25T19:02:45.199236+0000","last_peered":"2024-04-25T19:02:45.199236+0000","last_clean":"2024-04-25T19:02:45.199236+0000","last_became_active":"2024-04-25T19:02:40.075690+0000","last_became_peered":"2024-04-25T19:02:40.075690+0000","last_unstale":"2024-04-25T19:02:45.199236+0000","last_undegraded":"2024-04-25T19:02:45.199236+0000","last_fullsized":"2024-04-25T19:02:45.199236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202228+0000","last_change":"2024-04-25T19:02:40.075377+0000","last_active":"2024-04-25T19:02:45.202228+0000","last_peered":"2024-04-25T19:02:45.202228+0000","last_clean":"2024-04-25T19:02:45.202228+0000","last_became_active":"2024-04-25T19:02:40.069488+0000","last_became_peered":"2024-04-25T19:02:40.069488+0000","last_unstale":"2024-04-25T19:02:45.202228+0000","last_undegraded":"2024-04-25T19:02:45.202228+0000","last_fullsized":"2024-04-25T19:02:45.202228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128848+0000","last_change":"2024-04-25T19:02:40.083952+0000","last_active":"2024-04-25T19:02:45.128848+0000","last_peered":"2024-04-25T19:02:45.128848+0000","last_clean":"2024-04-25T19:02:45.128848+0000","last_became_active":"2024-04-25T19:02:40.083612+0000","last_became_peered":"2024-04-25T19:02:40.083612+0000","last_unstale":"2024-04-25T19:02:45.128848+0000","last_undegraded":"2024-04-25T19:02:45.128848+0000","last_fullsized":"2024-04-25T19:02:45.128848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199584+0000","last_change":"2024-04-25T19:02:40.067612+0000","last_active":"2024-04-25T19:02:45.199584+0000","last_peered":"2024-04-25T19:02:45.199584+0000","last_clean":"2024-04-25T19:02:45.199584+0000","last_became_active":"2024-04-25T19:02:40.067294+0000","last_became_peered":"2024-04-25T19:02:40.067294+0000","last_unstale":"2024-04-25T19:02:45.199584+0000","last_undegraded":"2024-04-25T19:02:45.199584+0000","last_fullsized":"2024-04-25T19:02:45.199584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197462+0000","last_change":"2024-04-25T19:02:40.073442+0000","last_active":"2024-04-25T19:02:45.197462+0000","last_peered":"2024-04-25T19:02:45.197462+0000","last_clean":"2024-04-25T19:02:45.197462+0000","last_became_active":"2024-04-25T19:02:40.073045+0000","last_became_peered":"2024-04-25T19:02:40.073045+0000","last_unstale":"2024-04-25T19:02:45.197462+0000","last_undegraded":"2024-04-25T19:02:45.197462+0000","last_fullsized":"2024-04-25T19:02:45.197462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131761+0000","last_change":"2024-04-25T19:02:40.050494+0000","last_active":"2024-04-25T19:02:45.131761+0000","last_peered":"2024-04-25T19:02:45.131761+0000","last_clean":"2024-04-25T19:02:45.131761+0000","last_became_active":"2024-04-25T19:02:40.050067+0000","last_became_peered":"2024-04-25T19:02:40.050067+0000","last_unstale":"2024-04-25T19:02:45.131761+0000","last_undegraded":"2024-04-25T19:02:45.131761+0000","last_fullsized":"2024-04-25T19:02:45.131761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198872+0000","last_change":"2024-04-25T19:02:40.071347+0000","last_active":"2024-04-25T19:02:45.198872+0000","last_peered":"2024-04-25T19:02:45.198872+0000","last_clean":"2024-04-25T19:02:45.198872+0000","last_became_active":"2024-04-25T19:02:40.071062+0000","last_became_peered":"2024-04-25T19:02:40.071062+0000","last_unstale":"2024-04-25T19:02:45.198872+0000","last_undegraded":"2024-04-25T19:02:45.198872+0000","last_fullsized":"2024-04-25T19:02:45.198872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130821+0000","last_change":"2024-04-25T19:02:40.076710+0000","last_active":"2024-04-25T19:02:45.130821+0000","last_peered":"2024-04-25T19:02:45.130821+0000","last_clean":"2024-04-25T19:02:45.130821+0000","last_became_active":"2024-04-25T19:02:40.071132+0000","last_became_peered":"2024-04-25T19:02:40.071132+0000","last_unstale":"2024-04-25T19:02:45.130821+0000","last_undegraded":"2024-04-25T19:02:45.130821+0000","last_fullsized":"2024-04-25T19:02:45.130821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:21:56.124743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202755+0000","last_change":"2024-04-25T19:02:40.076746+0000","last_active":"2024-04-25T19:02:45.202755+0000","last_peered":"2024-04-25T19:02:45.202755+0000","last_clean":"2024-04-25T19:02:45.202755+0000","last_became_active":"2024-04-25T19:02:40.068835+0000","last_became_peered":"2024-04-25T19:02:40.068835+0000","last_unstale":"2024-04-25T19:02:45.202755+0000","last_undegraded":"2024-04-25T19:02:45.202755+0000","last_fullsized":"2024-04-25T19:02:45.202755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132119+0000","last_change":"2024-04-25T19:02:40.090029+0000","last_active":"2024-04-25T19:02:45.132119+0000","last_peered":"2024-04-25T19:02:45.132119+0000","last_clean":"2024-04-25T19:02:45.132119+0000","last_became_active":"2024-04-25T19:02:40.088886+0000","last_became_peered":"2024-04-25T19:02:40.088886+0000","last_unstale":"2024-04-25T19:02:45.132119+0000","last_undegraded":"2024-04-25T19:02:45.132119+0000","last_fullsized":"2024-04-25T19:02:45.132119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:38.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128770+0000","last_change":"2024-04-25T19:02:40.079792+0000","last_active":"2024-04-25T19:02:45.128770+0000","last_peered":"2024-04-25T19:02:45.128770+0000","last_clean":"2024-04-25T19:02:45.128770+0000","last_became_active":"2024-04-25T19:02:40.071455+0000","last_became_peered":"2024-04-25T19:02:40.071455+0000","last_unstale":"2024-04-25T19:02:45.128770+0000","last_undegraded":"2024-04-25T19:02:45.128770+0000","last_fullsized":"2024-04-25T19:02:45.128770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198995+0000","last_change":"2024-04-25T19:02:40.088240+0000","last_active":"2024-04-25T19:02:45.198995+0000","last_peered":"2024-04-25T19:02:45.198995+0000","last_clean":"2024-04-25T19:02:45.198995+0000","last_became_active":"2024-04-25T19:02:40.087892+0000","last_became_peered":"2024-04-25T19:02:40.087892+0000","last_unstale":"2024-04-25T19:02:45.198995+0000","last_undegraded":"2024-04-25T19:02:45.198995+0000","last_fullsized":"2024-04-25T19:02:45.198995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133970+0000","last_change":"2024-04-25T19:02:40.062612+0000","last_active":"2024-04-25T19:02:45.133970+0000","last_peered":"2024-04-25T19:02:45.133970+0000","last_clean":"2024-04-25T19:02:45.133970+0000","last_became_active":"2024-04-25T19:02:40.060475+0000","last_became_peered":"2024-04-25T19:02:40.060475+0000","last_unstale":"2024-04-25T19:02:45.133970+0000","last_undegraded":"2024-04-25T19:02:45.133970+0000","last_fullsized":"2024-04-25T19:02:45.133970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:10.198621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135782+0000","last_change":"2024-04-25T19:02:40.091050+0000","last_active":"2024-04-25T19:02:45.135782+0000","last_peered":"2024-04-25T19:02:45.135782+0000","last_clean":"2024-04-25T19:02:45.135782+0000","last_became_active":"2024-04-25T19:02:40.090640+0000","last_became_peered":"2024-04-25T19:02:40.090640+0000","last_unstale":"2024-04-25T19:02:45.135782+0000","last_undegraded":"2024-04-25T19:02:45.135782+0000","last_fullsized":"2024-04-25T19:02:45.135782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:57.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198750+0000","last_change":"2024-04-25T19:02:40.074894+0000","last_active":"2024-04-25T19:02:45.198750+0000","last_peered":"2024-04-25T19:02:45.198750+0000","last_clean":"2024-04-25T19:02:45.198750+0000","last_became_active":"2024-04-25T19:02:40.074142+0000","last_became_peered":"2024-04-25T19:02:40.074142+0000","last_unstale":"2024-04-25T19:02:45.198750+0000","last_undegraded":"2024-04-25T19:02:45.198750+0000","last_fullsized":"2024-04-25T19:02:45.198750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198798+0000","last_change":"2024-04-25T19:02:40.087831+0000","last_active":"2024-04-25T19:02:45.198798+0000","last_peered":"2024-04-25T19:02:45.198798+0000","last_clean":"2024-04-25T19:02:45.198798+0000","last_became_active":"2024-04-25T19:02:40.087495+0000","last_became_peered":"2024-04-25T19:02:40.087495+0000","last_unstale":"2024-04-25T19:02:45.198798+0000","last_undegraded":"2024-04-25T19:02:45.198798+0000","last_fullsized":"2024-04-25T19:02:45.198798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197993+0000","last_change":"2024-04-25T19:02:40.067092+0000","last_active":"2024-04-25T19:02:45.197993+0000","last_peered":"2024-04-25T19:02:45.197993+0000","last_clean":"2024-04-25T19:02:45.197993+0000","last_became_active":"2024-04-25T19:02:40.066752+0000","last_became_peered":"2024-04-25T19:02:40.066752+0000","last_unstale":"2024-04-25T19:02:45.197993+0000","last_undegraded":"2024-04-25T19:02:45.197993+0000","last_fullsized":"2024-04-25T19:02:45.197993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200560+0000","last_change":"2024-04-25T19:02:40.069596+0000","last_active":"2024-04-25T19:02:45.200560+0000","last_peered":"2024-04-25T19:02:45.200560+0000","last_clean":"2024-04-25T19:02:45.200560+0000","last_became_active":"2024-04-25T19:02:40.069272+0000","last_became_peered":"2024-04-25T19:02:40.069272+0000","last_unstale":"2024-04-25T19:02:45.200560+0000","last_undegraded":"2024-04-25T19:02:45.200560+0000","last_fullsized":"2024-04-25T19:02:45.200560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198925+0000","last_change":"2024-04-25T19:02:40.085542+0000","last_active":"2024-04-25T19:02:45.198925+0000","last_peered":"2024-04-25T19:02:45.198925+0000","last_clean":"2024-04-25T19:02:45.198925+0000","last_became_active":"2024-04-25T19:02:40.085203+0000","last_became_peered":"2024-04-25T19:02:40.085203+0000","last_unstale":"2024-04-25T19:02:45.198925+0000","last_undegraded":"2024-04-25T19:02:45.198925+0000","last_fullsized":"2024-04-25T19:02:45.198925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197180+0000","last_change":"2024-04-25T19:02:40.070062+0000","last_active":"2024-04-25T19:02:45.197180+0000","last_peered":"2024-04-25T19:02:45.197180+0000","last_clean":"2024-04-25T19:02:45.197180+0000","last_became_active":"2024-04-25T19:02:40.069224+0000","last_became_peered":"2024-04-25T19:02:40.069224+0000","last_unstale":"2024-04-25T19:02:45.197180+0000","last_undegraded":"2024-04-25T19:02:45.197180+0000","last_fullsized":"2024-04-25T19:02:45.197180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198254+0000","last_change":"2024-04-25T19:02:40.055714+0000","last_active":"2024-04-25T19:02:45.198254+0000","last_peered":"2024-04-25T19:02:45.198254+0000","last_clean":"2024-04-25T19:02:45.198254+0000","last_became_active":"2024-04-25T19:02:40.052418+0000","last_became_peered":"2024-04-25T19:02:40.052418+0000","last_unstale":"2024-04-25T19:02:45.198254+0000","last_undegraded":"2024-04-25T19:02:45.198254+0000","last_fullsized":"2024-04-25T19:02:45.198254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198608+0000","last_change":"2024-04-25T19:02:40.087716+0000","last_active":"2024-04-25T19:02:45.198608+0000","last_peered":"2024-04-25T19:02:45.198608+0000","last_clean":"2024-04-25T19:02:45.198608+0000","last_became_active":"2024-04-25T19:02:40.085461+0000","last_became_peered":"2024-04-25T19:02:40.085461+0000","last_unstale":"2024-04-25T19:02:45.198608+0000","last_undegraded":"2024-04-25T19:02:45.198608+0000","last_fullsized":"2024-04-25T19:02:45.198608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198153+0000","last_change":"2024-04-25T19:02:40.053676+0000","last_active":"2024-04-25T19:02:45.198153+0000","last_peered":"2024-04-25T19:02:45.198153+0000","last_clean":"2024-04-25T19:02:45.198153+0000","last_became_active":"2024-04-25T19:02:40.052518+0000","last_became_peered":"2024-04-25T19:02:40.052518+0000","last_unstale":"2024-04-25T19:02:45.198153+0000","last_undegraded":"2024-04-25T19:02:45.198153+0000","last_fullsized":"2024-04-25T19:02:45.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:54.192825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201739+0000","last_change":"2024-04-25T19:02:40.076362+0000","last_active":"2024-04-25T19:02:45.201739+0000","last_peered":"2024-04-25T19:02:45.201739+0000","last_clean":"2024-04-25T19:02:45.201739+0000","last_became_active":"2024-04-25T19:02:40.070187+0000","last_became_peered":"2024-04-25T19:02:40.070187+0000","last_unstale":"2024-04-25T19:02:45.201739+0000","last_undegraded":"2024-04-25T19:02:45.201739+0000","last_fullsized":"2024-04-25T19:02:45.201739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202327+0000","last_change":"2024-04-25T19:02:40.072359+0000","last_active":"2024-04-25T19:02:45.202327+0000","last_peered":"2024-04-25T19:02:45.202327+0000","last_clean":"2024-04-25T19:02:45.202327+0000","last_became_active":"2024-04-25T19:02:40.071678+0000","last_became_peered":"2024-04-25T19:02:40.071678+0000","last_unstale":"2024-04-25T19:02:45.202327+0000","last_undegraded":"2024-04-25T19:02:45.202327+0000","last_fullsized":"2024-04-25T19:02:45.202327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198720+0000","last_change":"2024-04-25T19:02:40.078568+0000","last_active":"2024-04-25T19:02:45.198720+0000","last_peered":"2024-04-25T19:02:45.198720+0000","last_clean":"2024-04-25T19:02:45.198720+0000","last_became_active":"2024-04-25T19:02:40.078287+0000","last_became_peered":"2024-04-25T19:02:40.078287+0000","last_unstale":"2024-04-25T19:02:45.198720+0000","last_undegraded":"2024-04-25T19:02:45.198720+0000","last_fullsized":"2024-04-25T19:02:45.198720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:11.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:03:11.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.491+0000 7f338bc00640 1 Processor -- start 2024-04-25T19:03:11.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.491+0000 7f338bc00640 1 -- start start 2024-04-25T19:03:11.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.493+0000 7f338bc00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 0x7f3384070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.493+0000 7f338bc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3384154950 0x7f3384154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.493+0000 7f338bc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33840aec30 con 0x7f3384161b10 2024-04-25T19:03:11.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.493+0000 7f338bc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f33840d1530 con 0x7f3384154950 2024-04-25T19:03:11.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.493+0000 7f338bc00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f33840aef10 con 0x7f33840d1120 2024-04-25T19:03:11.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3384154950 0x7f3384154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3383e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3384161b10 0x7f3384161f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45334/0 (socket says 172.21.15.123:45334) 2024-04-25T19:03:11.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3383e00640 1 -- 172.21.15.123:0/568759861 learned_addr learned my addr 172.21.15.123:0/568759861 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389200640 1 --2- 172.21.15.123:0/568759861 >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 0x7f3384070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389c00640 1 -- 172.21.15.123:0/568759861 >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 msgr2=0x7f3384070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389c00640 1 --2- 172.21.15.123:0/568759861 >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 0x7f3384070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389c00640 1 -- 172.21.15.123:0/568759861 >> v1:172.21.15.123:6789/0 conn(0x7f3384161b10 legacy=0x7f3384161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389c00640 1 -- 172.21.15.123:0/568759861 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33840af1f0 con 0x7f3384154950 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3383400640 1 -- 172.21.15.123:0/568759861 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847787930 0 0) 0x7f33840aec30 con 0x7f3384161b10 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.494+0000 7f3389200640 1 --2- 172.21.15.123:0/568759861 >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 0x7f3384070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:11.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.495+0000 7f3389c00640 1 --2- 172.21.15.123:0/568759861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3384154950 0x7f3384154d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33840b6a60 tx=0x7f337c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11269536b382587f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:11.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.495+0000 7f3383400640 1 -- 172.21.15.123:0/568759861 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c007740 con 0x7f3384154950 2024-04-25T19:03:11.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.495+0000 7f3383400640 1 -- 172.21.15.123:0/568759861 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337c0078e0 con 0x7f3384154950 2024-04-25T19:03:11.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f3383400640 1 -- 172.21.15.123:0/568759861 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c007cb0 con 0x7f3384154950 2024-04-25T19:03:11.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 -- 172.21.15.123:0/568759861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3384154950 msgr2=0x7f3384154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 --2- 172.21.15.123:0/568759861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3384154950 0x7f3384154d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33840b6a60 tx=0x7f337c02a560 comp rx=0 tx=0).stop 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 -- 172.21.15.123:0/568759861 shutdown_connections 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 --2- 172.21.15.123:0/568759861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3384154950 0x7f3384154d60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 --2- 172.21.15.123:0/568759861 >> v2:172.21.15.146:3300/0 conn(0x7f33840d1120 0x7f3384070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 -- 172.21.15.123:0/568759861 >> 172.21.15.123:0/568759861 conn(0x7f33840d1ae0 msgr2=0x7f33840d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 -- 172.21.15.123:0/568759861 shutdown_connections 2024-04-25T19:03:11.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.496+0000 7f338bc00640 1 -- 172.21.15.123:0/568759861 wait complete. 2024-04-25T19:03:11.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.499+0000 7f338bc00640 1 Processor -- start 2024-04-25T19:03:11.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.499+0000 7f338bc00640 1 -- start start 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3384154950 0x7f33841524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 0x7f33841f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33841552a0 con 0x7f33840d1120 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3384071430 con 0x7f3384161b10 2024-04-25T19:03:11.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f338bc00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3384159870 con 0x7f3384154950 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f3389200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f3389200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34514/0 (socket says 172.21.15.123:34514) 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.500+0000 7f3389200640 1 -- 172.21.15.123:0/525018320 learned_addr learned my addr 172.21.15.123:0/525018320 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389c00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 0x7f33841f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3383e00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3384154950 0x7f33841524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 -- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3384154950 msgr2=0x7f33841524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3384154950 0x7f33841524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 -- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 msgr2=0x7f33841f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 0x7f33841f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 -- 172.21.15.123:0/525018320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3384152c50 con 0x7f33840d1120 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389c00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 0x7f33841f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3389200640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f337800c820 tx=0x7f337800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378015070 con 0x7f33840d1120 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3378007a30 con 0x7f33840d1120 2024-04-25T19:03:11.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378007de0 con 0x7f33840d1120 2024-04-25T19:03:11.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.501+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3384152f30 con 0x7f33840d1120 2024-04-25T19:03:11.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.502+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841e9780 con 0x7f33840d1120 2024-04-25T19:03:11.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.503+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3384076a40 con 0x7f33840d1120 2024-04-25T19:03:11.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.504+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3378018070 con 0x7f33840d1120 2024-04-25T19:03:11.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.505+0000 7f3380c00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 0x7f33640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:11.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.505+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f33780954e0 con 0x7f33840d1120 2024-04-25T19:03:11.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.505+0000 7f3383e00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 0x7f33640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:11.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.506+0000 7f3383e00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 0x7f33640769f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f338406dbb0 tx=0x7f3370009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:11.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.509+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f337805f920 con 0x7f33840d1120 2024-04-25T19:03:11.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.693+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33841e9cf0 con 0x7f33640744e0 2024-04-25T19:03:11.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.708+0000 7f3380c00640 1 -- 172.21.15.123:0/525018320 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332522 (secure 0 0 0) 0x7f33841e9cf0 con 0x7f33640744e0 2024-04-25T19:03:11.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:11.716 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 msgr2=0x7f33640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 0x7f33640769f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f338406dbb0 tx=0x7f3370009040 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 msgr2=0x7f3384153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f337800c820 tx=0x7f337800cce0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 shutdown_connections 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f33640744e0 0x7f33640769f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3384161b10 0x7f33841f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3384154950 0x7f33841524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 --2- 172.21.15.123:0/525018320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33840d1120 0x7f3384153a10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.715+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 >> 172.21.15.123:0/525018320 conn(0x7f33840d1ae0 msgr2=0x7f33840b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:11.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.716+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 shutdown_connections 2024-04-25T19:03:11.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:11.716+0000 7f338bc00640 1 -- 172.21.15.123:0/525018320 wait complete. 2024-04-25T19:03:11.741 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T19:03:10.973164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219516,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755204,"statfs":{"total":773094113280,"available":772869328896,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005398"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137574+0000","last_change":"2024-04-25T19:02:39.052814+0000","last_active":"2024-04-25T19:02:45.137574+0000","last_peered":"2024-04-25T19:02:45.137574+0000","last_clean":"2024-04-25T19:02:45.137574+0000","last_became_active":"2024-04-25T19:02:39.052399+0000","last_became_peered":"2024-04-25T19:02:39.052399+0000","last_unstale":"2024-04-25T19:02:45.137574+0000","last_undegraded":"2024-04-25T19:02:45.137574+0000","last_fullsized":"2024-04-25T19:02:45.137574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202717+0000","last_change":"2024-04-25T19:02:40.072630+0000","last_active":"2024-04-25T19:02:45.202717+0000","last_peered":"2024-04-25T19:02:45.202717+0000","last_clean":"2024-04-25T19:02:45.202717+0000","last_became_active":"2024-04-25T19:02:40.072046+0000","last_became_peered":"2024-04-25T19:02:40.072046+0000","last_unstale":"2024-04-25T19:02:45.202717+0000","last_undegraded":"2024-04-25T19:02:45.202717+0000","last_fullsized":"2024-04-25T19:02:45.202717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200723+0000","last_change":"2024-04-25T19:02:44.161105+0000","last_active":"2024-04-25T19:02:45.200723+0000","last_peered":"2024-04-25T19:02:45.200723+0000","last_clean":"2024-04-25T19:02:45.200723+0000","last_became_active":"2024-04-25T19:02:44.160613+0000","last_became_peered":"2024-04-25T19:02:44.160613+0000","last_unstale":"2024-04-25T19:02:45.200723+0000","last_undegraded":"2024-04-25T19:02:45.200723+0000","last_fullsized":"2024-04-25T19:02:45.200723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:38.319660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203032+0000","last_change":"2024-04-25T19:02:39.042391+0000","last_active":"2024-04-25T19:02:45.203032+0000","last_peered":"2024-04-25T19:02:45.203032+0000","last_clean":"2024-04-25T19:02:45.203032+0000","last_became_active":"2024-04-25T19:02:39.042017+0000","last_became_peered":"2024-04-25T19:02:39.042017+0000","last_unstale":"2024-04-25T19:02:45.203032+0000","last_undegraded":"2024-04-25T19:02:45.203032+0000","last_fullsized":"2024-04-25T19:02:45.203032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203099+0000","last_change":"2024-04-25T19:02:40.073983+0000","last_active":"2024-04-25T19:02:45.203099+0000","last_peered":"2024-04-25T19:02:45.203099+0000","last_clean":"2024-04-25T19:02:45.203099+0000","last_became_active":"2024-04-25T19:02:40.069080+0000","last_became_peered":"2024-04-25T19:02:40.069080+0000","last_unstale":"2024-04-25T19:02:45.203099+0000","last_undegraded":"2024-04-25T19:02:45.203099+0000","last_fullsized":"2024-04-25T19:02:45.203099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198316+0000","last_change":"2024-04-25T19:02:44.148887+0000","last_active":"2024-04-25T19:02:45.198316+0000","last_peered":"2024-04-25T19:02:45.198316+0000","last_clean":"2024-04-25T19:02:45.198316+0000","last_became_active":"2024-04-25T19:02:44.148199+0000","last_became_peered":"2024-04-25T19:02:44.148199+0000","last_unstale":"2024-04-25T19:02:45.198316+0000","last_undegraded":"2024-04-25T19:02:45.198316+0000","last_fullsized":"2024-04-25T19:02:45.198316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:55.854080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199269+0000","last_change":"2024-04-25T19:02:39.047541+0000","last_active":"2024-04-25T19:02:45.199269+0000","last_peered":"2024-04-25T19:02:45.199269+0000","last_clean":"2024-04-25T19:02:45.199269+0000","last_became_active":"2024-04-25T19:02:39.047208+0000","last_became_peered":"2024-04-25T19:02:39.047208+0000","last_unstale":"2024-04-25T19:02:45.199269+0000","last_undegraded":"2024-04-25T19:02:45.199269+0000","last_fullsized":"2024-04-25T19:02:45.199269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200023+0000","last_change":"2024-04-25T19:02:40.076534+0000","last_active":"2024-04-25T19:02:45.200023+0000","last_peered":"2024-04-25T19:02:45.200023+0000","last_clean":"2024-04-25T19:02:45.200023+0000","last_became_active":"2024-04-25T19:02:40.076129+0000","last_became_peered":"2024-04-25T19:02:40.076129+0000","last_unstale":"2024-04-25T19:02:45.200023+0000","last_undegraded":"2024-04-25T19:02:45.200023+0000","last_fullsized":"2024-04-25T19:02:45.200023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198683+0000","last_change":"2024-04-25T19:02:44.153635+0000","last_active":"2024-04-25T19:02:45.198683+0000","last_peered":"2024-04-25T19:02:45.198683+0000","last_clean":"2024-04-25T19:02:45.198683+0000","last_became_active":"2024-04-25T19:02:44.153134+0000","last_became_peered":"2024-04-25T19:02:44.153134+0000","last_unstale":"2024-04-25T19:02:45.198683+0000","last_undegraded":"2024-04-25T19:02:45.198683+0000","last_fullsized":"2024-04-25T19:02:45.198683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:25.102590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199200+0000","last_change":"2024-04-25T19:02:39.047906+0000","last_active":"2024-04-25T19:02:45.199200+0000","last_peered":"2024-04-25T19:02:45.199200+0000","last_clean":"2024-04-25T19:02:45.199200+0000","last_became_active":"2024-04-25T19:02:39.044722+0000","last_became_peered":"2024-04-25T19:02:39.044722+0000","last_unstale":"2024-04-25T19:02:45.199200+0000","last_undegraded":"2024-04-25T19:02:45.199200+0000","last_fullsized":"2024-04-25T19:02:45.199200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130532+0000","last_change":"2024-04-25T19:02:40.084937+0000","last_active":"2024-04-25T19:02:45.130532+0000","last_peered":"2024-04-25T19:02:45.130532+0000","last_clean":"2024-04-25T19:02:45.130532+0000","last_became_active":"2024-04-25T19:02:40.084547+0000","last_became_peered":"2024-04-25T19:02:40.084547+0000","last_unstale":"2024-04-25T19:02:45.130532+0000","last_undegraded":"2024-04-25T19:02:45.130532+0000","last_fullsized":"2024-04-25T19:02:45.130532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:00.967270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197619+0000","last_change":"2024-04-25T19:02:44.152662+0000","last_active":"2024-04-25T19:02:45.197619+0000","last_peered":"2024-04-25T19:02:45.197619+0000","last_clean":"2024-04-25T19:02:45.197619+0000","last_became_active":"2024-04-25T19:02:44.152150+0000","last_became_peered":"2024-04-25T19:02:44.152150+0000","last_unstale":"2024-04-25T19:02:45.197619+0000","last_undegraded":"2024-04-25T19:02:45.197619+0000","last_fullsized":"2024-04-25T19:02:45.197619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:04.766506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129234+0000","last_change":"2024-04-25T19:02:40.069483+0000","last_active":"2024-04-25T19:02:45.129234+0000","last_peered":"2024-04-25T19:02:45.129234+0000","last_clean":"2024-04-25T19:02:45.129234+0000","last_became_active":"2024-04-25T19:02:40.068633+0000","last_became_peered":"2024-04-25T19:02:40.068633+0000","last_unstale":"2024-04-25T19:02:45.129234+0000","last_undegraded":"2024-04-25T19:02:45.129234+0000","last_fullsized":"2024-04-25T19:02:45.129234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:46.741238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130329+0000","last_change":"2024-04-25T19:02:39.036777+0000","last_active":"2024-04-25T19:02:45.130329+0000","last_peered":"2024-04-25T19:02:45.130329+0000","last_clean":"2024-04-25T19:02:45.130329+0000","last_became_active":"2024-04-25T19:02:39.036471+0000","last_became_peered":"2024-04-25T19:02:39.036471+0000","last_unstale":"2024-04-25T19:02:45.130329+0000","last_undegraded":"2024-04-25T19:02:45.130329+0000","last_fullsized":"2024-04-25T19:02:45.130329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:41.460879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130171+0000","last_change":"2024-04-25T19:02:44.153484+0000","last_active":"2024-04-25T19:02:45.130171+0000","last_peered":"2024-04-25T19:02:45.130171+0000","last_clean":"2024-04-25T19:02:45.130171+0000","last_became_active":"2024-04-25T19:02:44.153061+0000","last_became_peered":"2024-04-25T19:02:44.153061+0000","last_unstale":"2024-04-25T19:02:45.130171+0000","last_undegraded":"2024-04-25T19:02:45.130171+0000","last_fullsized":"2024-04-25T19:02:45.130171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:14.106695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201441+0000","last_change":"2024-04-25T19:02:40.071684+0000","last_active":"2024-04-25T19:02:45.201441+0000","last_peered":"2024-04-25T19:02:45.201441+0000","last_clean":"2024-04-25T19:02:45.201441+0000","last_became_active":"2024-04-25T19:02:40.071149+0000","last_became_peered":"2024-04-25T19:02:40.071149+0000","last_unstale":"2024-04-25T19:02:45.201441+0000","last_undegraded":"2024-04-25T19:02:45.201441+0000","last_fullsized":"2024-04-25T19:02:45.201441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134616+0000","last_change":"2024-04-25T19:02:39.047265+0000","last_active":"2024-04-25T19:02:45.134616+0000","last_peered":"2024-04-25T19:02:45.134616+0000","last_clean":"2024-04-25T19:02:45.134616+0000","last_became_active":"2024-04-25T19:02:39.046336+0000","last_became_peered":"2024-04-25T19:02:39.046336+0000","last_unstale":"2024-04-25T19:02:45.134616+0000","last_undegraded":"2024-04-25T19:02:45.134616+0000","last_fullsized":"2024-04-25T19:02:45.134616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:02.247449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198218+0000","last_change":"2024-04-25T19:02:44.139798+0000","last_active":"2024-04-25T19:02:45.198218+0000","last_peered":"2024-04-25T19:02:45.198218+0000","last_clean":"2024-04-25T19:02:45.198218+0000","last_became_active":"2024-04-25T19:02:44.139333+0000","last_became_peered":"2024-04-25T19:02:44.139333+0000","last_unstale":"2024-04-25T19:02:45.198218+0000","last_undegraded":"2024-04-25T19:02:45.198218+0000","last_fullsized":"2024-04-25T19:02:45.198218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:05.053484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131214+0000","last_change":"2024-04-25T19:02:40.065574+0000","last_active":"2024-04-25T19:02:45.131214+0000","last_peered":"2024-04-25T19:02:45.131214+0000","last_clean":"2024-04-25T19:02:45.131214+0000","last_became_active":"2024-04-25T19:02:40.060953+0000","last_became_peered":"2024-04-25T19:02:40.060953+0000","last_unstale":"2024-04-25T19:02:45.131214+0000","last_undegraded":"2024-04-25T19:02:45.131214+0000","last_fullsized":"2024-04-25T19:02:45.131214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132354+0000","last_change":"2024-04-25T19:02:39.044906+0000","last_active":"2024-04-25T19:02:45.132354+0000","last_peered":"2024-04-25T19:02:45.132354+0000","last_clean":"2024-04-25T19:02:45.132354+0000","last_became_active":"2024-04-25T19:02:39.043065+0000","last_became_peered":"2024-04-25T19:02:39.043065+0000","last_unstale":"2024-04-25T19:02:45.132354+0000","last_undegraded":"2024-04-25T19:02:45.132354+0000","last_fullsized":"2024-04-25T19:02:45.132354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203780+0000","last_change":"2024-04-25T19:02:44.149122+0000","last_active":"2024-04-25T19:02:45.203780+0000","last_peered":"2024-04-25T19:02:45.203780+0000","last_clean":"2024-04-25T19:02:45.203780+0000","last_became_active":"2024-04-25T19:02:44.148633+0000","last_became_peered":"2024-04-25T19:02:44.148633+0000","last_unstale":"2024-04-25T19:02:45.203780+0000","last_undegraded":"2024-04-25T19:02:45.203780+0000","last_fullsized":"2024-04-25T19:02:45.203780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128923+0000","last_change":"2024-04-25T19:02:40.067343+0000","last_active":"2024-04-25T19:02:45.128923+0000","last_peered":"2024-04-25T19:02:45.128923+0000","last_clean":"2024-04-25T19:02:45.128923+0000","last_became_active":"2024-04-25T19:02:40.066748+0000","last_became_peered":"2024-04-25T19:02:40.066748+0000","last_unstale":"2024-04-25T19:02:45.128923+0000","last_undegraded":"2024-04-25T19:02:45.128923+0000","last_fullsized":"2024-04-25T19:02:45.128923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:18.495508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196634+0000","last_change":"2024-04-25T19:02:39.042622+0000","last_active":"2024-04-25T19:02:45.196634+0000","last_peered":"2024-04-25T19:02:45.196634+0000","last_clean":"2024-04-25T19:02:45.196634+0000","last_became_active":"2024-04-25T19:02:39.042302+0000","last_became_peered":"2024-04-25T19:02:39.042302+0000","last_unstale":"2024-04-25T19:02:45.196634+0000","last_undegraded":"2024-04-25T19:02:45.196634+0000","last_fullsized":"2024-04-25T19:02:45.196634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199162+0000","last_change":"2024-04-25T19:02:44.148027+0000","last_active":"2024-04-25T19:02:45.199162+0000","last_peered":"2024-04-25T19:02:45.199162+0000","last_clean":"2024-04-25T19:02:45.199162+0000","last_became_active":"2024-04-25T19:02:44.147612+0000","last_became_peered":"2024-04-25T19:02:44.147612+0000","last_unstale":"2024-04-25T19:02:45.199162+0000","last_undegraded":"2024-04-25T19:02:45.199162+0000","last_fullsized":"2024-04-25T19:02:45.199162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:52.678990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136112+0000","last_change":"2024-04-25T19:02:40.062872+0000","last_active":"2024-04-25T19:02:45.136112+0000","last_peered":"2024-04-25T19:02:45.136112+0000","last_clean":"2024-04-25T19:02:45.136112+0000","last_became_active":"2024-04-25T19:02:40.061519+0000","last_became_peered":"2024-04-25T19:02:40.061519+0000","last_unstale":"2024-04-25T19:02:45.136112+0000","last_undegraded":"2024-04-25T19:02:45.136112+0000","last_fullsized":"2024-04-25T19:02:45.136112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132193+0000","last_change":"2024-04-25T19:02:39.054460+0000","last_active":"2024-04-25T19:02:45.132193+0000","last_peered":"2024-04-25T19:02:45.132193+0000","last_clean":"2024-04-25T19:02:45.132193+0000","last_became_active":"2024-04-25T19:02:39.053233+0000","last_became_peered":"2024-04-25T19:02:39.053233+0000","last_unstale":"2024-04-25T19:02:45.132193+0000","last_undegraded":"2024-04-25T19:02:45.132193+0000","last_fullsized":"2024-04-25T19:02:45.132193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:17.622604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199607+0000","last_change":"2024-04-25T19:02:44.139268+0000","last_active":"2024-04-25T19:02:45.199607+0000","last_peered":"2024-04-25T19:02:45.199607+0000","last_clean":"2024-04-25T19:02:45.199607+0000","last_became_active":"2024-04-25T19:02:44.137641+0000","last_became_peered":"2024-04-25T19:02:44.137641+0000","last_unstale":"2024-04-25T19:02:45.199607+0000","last_undegraded":"2024-04-25T19:02:45.199607+0000","last_fullsized":"2024-04-25T19:02:45.199607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137364+0000","last_change":"2024-04-25T19:02:40.063483+0000","last_active":"2024-04-25T19:02:45.137364+0000","last_peered":"2024-04-25T19:02:45.137364+0000","last_clean":"2024-04-25T19:02:45.137364+0000","last_became_active":"2024-04-25T19:02:40.062531+0000","last_became_peered":"2024-04-25T19:02:40.062531+0000","last_unstale":"2024-04-25T19:02:45.137364+0000","last_undegraded":"2024-04-25T19:02:45.137364+0000","last_fullsized":"2024-04-25T19:02:45.137364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136358+0000","last_change":"2024-04-25T19:02:39.052714+0000","last_active":"2024-04-25T19:02:45.136358+0000","last_peered":"2024-04-25T19:02:45.136358+0000","last_clean":"2024-04-25T19:02:45.136358+0000","last_became_active":"2024-04-25T19:02:39.051543+0000","last_became_peered":"2024-04-25T19:02:39.051543+0000","last_unstale":"2024-04-25T19:02:45.136358+0000","last_undegraded":"2024-04-25T19:02:45.136358+0000","last_fullsized":"2024-04-25T19:02:45.136358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131504+0000","last_change":"2024-04-25T19:02:44.142273+0000","last_active":"2024-04-25T19:02:45.131504+0000","last_peered":"2024-04-25T19:02:45.131504+0000","last_clean":"2024-04-25T19:02:45.131504+0000","last_became_active":"2024-04-25T19:02:44.140876+0000","last_became_peered":"2024-04-25T19:02:44.140876+0000","last_unstale":"2024-04-25T19:02:45.131504+0000","last_undegraded":"2024-04-25T19:02:45.131504+0000","last_fullsized":"2024-04-25T19:02:45.131504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:07.994733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137451+0000","last_change":"2024-04-25T19:02:40.065602+0000","last_active":"2024-04-25T19:02:45.137451+0000","last_peered":"2024-04-25T19:02:45.137451+0000","last_clean":"2024-04-25T19:02:45.137451+0000","last_became_active":"2024-04-25T19:02:40.065178+0000","last_became_peered":"2024-04-25T19:02:40.065178+0000","last_unstale":"2024-04-25T19:02:45.137451+0000","last_undegraded":"2024-04-25T19:02:45.137451+0000","last_fullsized":"2024-04-25T19:02:45.137451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.101532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131078+0000","last_change":"2024-04-25T19:02:39.054975+0000","last_active":"2024-04-25T19:02:45.131078+0000","last_peered":"2024-04-25T19:02:45.131078+0000","last_clean":"2024-04-25T19:02:45.131078+0000","last_became_active":"2024-04-25T19:02:39.052591+0000","last_became_peered":"2024-04-25T19:02:39.052591+0000","last_unstale":"2024-04-25T19:02:45.131078+0000","last_undegraded":"2024-04-25T19:02:45.131078+0000","last_fullsized":"2024-04-25T19:02:45.131078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:10.725639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135617+0000","last_change":"2024-04-25T19:02:44.144565+0000","last_active":"2024-04-25T19:02:45.135617+0000","last_peered":"2024-04-25T19:02:45.135617+0000","last_clean":"2024-04-25T19:02:45.135617+0000","last_became_active":"2024-04-25T19:02:44.144092+0000","last_became_peered":"2024-04-25T19:02:44.144092+0000","last_unstale":"2024-04-25T19:02:45.135617+0000","last_undegraded":"2024-04-25T19:02:45.135617+0000","last_fullsized":"2024-04-25T19:02:45.135617+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:18.177897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198835+0000","last_change":"2024-04-25T19:02:40.062748+0000","last_active":"2024-04-25T19:02:45.198835+0000","last_peered":"2024-04-25T19:02:45.198835+0000","last_clean":"2024-04-25T19:02:45.198835+0000","last_became_active":"2024-04-25T19:02:40.060618+0000","last_became_peered":"2024-04-25T19:02:40.060618+0000","last_unstale":"2024-04-25T19:02:45.198835+0000","last_undegraded":"2024-04-25T19:02:45.198835+0000","last_fullsized":"2024-04-25T19:02:45.198835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200394+0000","last_change":"2024-04-25T19:02:39.043998+0000","last_active":"2024-04-25T19:02:45.200394+0000","last_peered":"2024-04-25T19:02:45.200394+0000","last_clean":"2024-04-25T19:02:45.200394+0000","last_became_active":"2024-04-25T19:02:39.043623+0000","last_became_peered":"2024-04-25T19:02:39.043623+0000","last_unstale":"2024-04-25T19:02:45.200394+0000","last_undegraded":"2024-04-25T19:02:45.200394+0000","last_fullsized":"2024-04-25T19:02:45.200394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133644+0000","last_change":"2024-04-25T19:02:44.158919+0000","last_active":"2024-04-25T19:02:45.133644+0000","last_peered":"2024-04-25T19:02:45.133644+0000","last_clean":"2024-04-25T19:02:45.133644+0000","last_became_active":"2024-04-25T19:02:44.158358+0000","last_became_peered":"2024-04-25T19:02:44.158358+0000","last_unstale":"2024-04-25T19:02:45.133644+0000","last_undegraded":"2024-04-25T19:02:45.133644+0000","last_fullsized":"2024-04-25T19:02:45.133644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:05.831945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198482+0000","last_change":"2024-04-25T19:02:40.072547+0000","last_active":"2024-04-25T19:02:45.198482+0000","last_peered":"2024-04-25T19:02:45.198482+0000","last_clean":"2024-04-25T19:02:45.198482+0000","last_became_active":"2024-04-25T19:02:40.072124+0000","last_became_peered":"2024-04-25T19:02:40.072124+0000","last_unstale":"2024-04-25T19:02:45.198482+0000","last_undegraded":"2024-04-25T19:02:45.198482+0000","last_fullsized":"2024-04-25T19:02:45.198482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130699+0000","last_change":"2024-04-25T19:02:39.051040+0000","last_active":"2024-04-25T19:02:45.130699+0000","last_peered":"2024-04-25T19:02:45.130699+0000","last_clean":"2024-04-25T19:02:45.130699+0000","last_became_active":"2024-04-25T19:02:39.048647+0000","last_became_peered":"2024-04-25T19:02:39.048647+0000","last_unstale":"2024-04-25T19:02:45.130699+0000","last_undegraded":"2024-04-25T19:02:45.130699+0000","last_fullsized":"2024-04-25T19:02:45.130699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:53.187342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136962+0000","last_change":"2024-04-25T19:02:44.151392+0000","last_active":"2024-04-25T19:02:45.136962+0000","last_peered":"2024-04-25T19:02:45.136962+0000","last_clean":"2024-04-25T19:02:45.136962+0000","last_became_active":"2024-04-25T19:02:44.150074+0000","last_became_peered":"2024-04-25T19:02:44.150074+0000","last_unstale":"2024-04-25T19:02:45.136962+0000","last_undegraded":"2024-04-25T19:02:45.136962+0000","last_fullsized":"2024-04-25T19:02:45.136962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:03.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199109+0000","last_change":"2024-04-25T19:02:40.074602+0000","last_active":"2024-04-25T19:02:45.199109+0000","last_peered":"2024-04-25T19:02:45.199109+0000","last_clean":"2024-04-25T19:02:45.199109+0000","last_became_active":"2024-04-25T19:02:40.070530+0000","last_became_peered":"2024-04-25T19:02:40.070530+0000","last_unstale":"2024-04-25T19:02:45.199109+0000","last_undegraded":"2024-04-25T19:02:45.199109+0000","last_fullsized":"2024-04-25T19:02:45.199109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:55.634101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200966+0000","last_change":"2024-04-25T19:02:39.048859+0000","last_active":"2024-04-25T19:02:45.200966+0000","last_peered":"2024-04-25T19:02:45.200966+0000","last_clean":"2024-04-25T19:02:45.200966+0000","last_became_active":"2024-04-25T19:02:39.048497+0000","last_became_peered":"2024-04-25T19:02:39.048497+0000","last_unstale":"2024-04-25T19:02:45.200966+0000","last_undegraded":"2024-04-25T19:02:45.200966+0000","last_fullsized":"2024-04-25T19:02:45.200966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:13.519715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137567+0000","last_change":"2024-04-25T19:02:44.150754+0000","last_active":"2024-04-25T19:02:45.137567+0000","last_peered":"2024-04-25T19:02:45.137567+0000","last_clean":"2024-04-25T19:02:45.137567+0000","last_became_active":"2024-04-25T19:02:44.150092+0000","last_became_peered":"2024-04-25T19:02:44.150092+0000","last_unstale":"2024-04-25T19:02:45.137567+0000","last_undegraded":"2024-04-25T19:02:45.137567+0000","last_fullsized":"2024-04-25T19:02:45.137567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:45.602374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132688+0000","last_change":"2024-04-25T19:02:40.068029+0000","last_active":"2024-04-25T19:02:45.132688+0000","last_peered":"2024-04-25T19:02:45.132688+0000","last_clean":"2024-04-25T19:02:45.132688+0000","last_became_active":"2024-04-25T19:02:40.066355+0000","last_became_peered":"2024-04-25T19:02:40.066355+0000","last_unstale":"2024-04-25T19:02:45.132688+0000","last_undegraded":"2024-04-25T19:02:45.132688+0000","last_fullsized":"2024-04-25T19:02:45.132688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:09.936302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129192+0000","last_change":"2024-04-25T19:02:39.047056+0000","last_active":"2024-04-25T19:02:45.129192+0000","last_peered":"2024-04-25T19:02:45.129192+0000","last_clean":"2024-04-25T19:02:45.129192+0000","last_became_active":"2024-04-25T19:02:39.046198+0000","last_became_peered":"2024-04-25T19:02:39.046198+0000","last_unstale":"2024-04-25T19:02:45.129192+0000","last_undegraded":"2024-04-25T19:02:45.129192+0000","last_fullsized":"2024-04-25T19:02:45.129192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:33.150948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202550+0000","last_change":"2024-04-25T19:02:44.158508+0000","last_active":"2024-04-25T19:02:45.202550+0000","last_peered":"2024-04-25T19:02:45.202550+0000","last_clean":"2024-04-25T19:02:45.202550+0000","last_became_active":"2024-04-25T19:02:44.157734+0000","last_became_peered":"2024-04-25T19:02:44.157734+0000","last_unstale":"2024-04-25T19:02:45.202550+0000","last_undegraded":"2024-04-25T19:02:45.202550+0000","last_fullsized":"2024-04-25T19:02:45.202550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:42.894635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133801+0000","last_change":"2024-04-25T19:02:40.048821+0000","last_active":"2024-04-25T19:02:45.133801+0000","last_peered":"2024-04-25T19:02:45.133801+0000","last_clean":"2024-04-25T19:02:45.133801+0000","last_became_active":"2024-04-25T19:02:40.048295+0000","last_became_peered":"2024-04-25T19:02:40.048295+0000","last_unstale":"2024-04-25T19:02:45.133801+0000","last_undegraded":"2024-04-25T19:02:45.133801+0000","last_fullsized":"2024-04-25T19:02:45.133801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129911+0000","last_change":"2024-04-25T19:02:39.036175+0000","last_active":"2024-04-25T19:02:45.129911+0000","last_peered":"2024-04-25T19:02:45.129911+0000","last_clean":"2024-04-25T19:02:45.129911+0000","last_became_active":"2024-04-25T19:02:39.035822+0000","last_became_peered":"2024-04-25T19:02:39.035822+0000","last_unstale":"2024-04-25T19:02:45.129911+0000","last_undegraded":"2024-04-25T19:02:45.129911+0000","last_fullsized":"2024-04-25T19:02:45.129911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.306938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197039+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:02:45.197039+0000","last_peered":"2024-04-25T19:02:45.197039+0000","last_clean":"2024-04-25T19:02:45.197039+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:02:45.197039+0000","last_undegraded":"2024-04-25T19:02:45.197039+0000","last_fullsized":"2024-04-25T19:02:45.197039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131234+0000","last_change":"2024-04-25T19:02:44.144534+0000","last_active":"2024-04-25T19:02:45.131234+0000","last_peered":"2024-04-25T19:02:45.131234+0000","last_clean":"2024-04-25T19:02:45.131234+0000","last_became_active":"2024-04-25T19:02:44.143365+0000","last_became_peered":"2024-04-25T19:02:44.143365+0000","last_unstale":"2024-04-25T19:02:45.131234+0000","last_undegraded":"2024-04-25T19:02:45.131234+0000","last_fullsized":"2024-04-25T19:02:45.131234+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:39.667305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130665+0000","last_change":"2024-04-25T19:02:40.082915+0000","last_active":"2024-04-25T19:02:45.130665+0000","last_peered":"2024-04-25T19:02:45.130665+0000","last_clean":"2024-04-25T19:02:45.130665+0000","last_became_active":"2024-04-25T19:02:40.082594+0000","last_became_peered":"2024-04-25T19:02:40.082594+0000","last_unstale":"2024-04-25T19:02:45.130665+0000","last_undegraded":"2024-04-25T19:02:45.130665+0000","last_fullsized":"2024-04-25T19:02:45.130665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:10.588191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137737+0000","last_change":"2024-04-25T19:02:39.052990+0000","last_active":"2024-04-25T19:02:45.137737+0000","last_peered":"2024-04-25T19:02:45.137737+0000","last_clean":"2024-04-25T19:02:45.137737+0000","last_became_active":"2024-04-25T19:02:39.052635+0000","last_became_peered":"2024-04-25T19:02:39.052635+0000","last_unstale":"2024-04-25T19:02:45.137737+0000","last_undegraded":"2024-04-25T19:02:45.137737+0000","last_fullsized":"2024-04-25T19:02:45.137737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.298378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198050+0000","last_change":"2024-04-25T19:02:44.143773+0000","last_active":"2024-04-25T19:02:45.198050+0000","last_peered":"2024-04-25T19:02:45.198050+0000","last_clean":"2024-04-25T19:02:45.198050+0000","last_became_active":"2024-04-25T19:02:44.141602+0000","last_became_peered":"2024-04-25T19:02:44.141602+0000","last_unstale":"2024-04-25T19:02:45.198050+0000","last_undegraded":"2024-04-25T19:02:45.198050+0000","last_fullsized":"2024-04-25T19:02:45.198050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:03.052112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199104+0000","last_change":"2024-04-25T19:02:40.075177+0000","last_active":"2024-04-25T19:02:45.199104+0000","last_peered":"2024-04-25T19:02:45.199104+0000","last_clean":"2024-04-25T19:02:45.199104+0000","last_became_active":"2024-04-25T19:02:40.074512+0000","last_became_peered":"2024-04-25T19:02:40.074512+0000","last_unstale":"2024-04-25T19:02:45.199104+0000","last_undegraded":"2024-04-25T19:02:45.199104+0000","last_fullsized":"2024-04-25T19:02:45.199104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201661+0000","last_change":"2024-04-25T19:02:39.052054+0000","last_active":"2024-04-25T19:02:45.201661+0000","last_peered":"2024-04-25T19:02:45.201661+0000","last_clean":"2024-04-25T19:02:45.201661+0000","last_became_active":"2024-04-25T19:02:39.051646+0000","last_became_peered":"2024-04-25T19:02:39.051646+0000","last_unstale":"2024-04-25T19:02:45.201661+0000","last_undegraded":"2024-04-25T19:02:45.201661+0000","last_fullsized":"2024-04-25T19:02:45.201661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200618+0000","last_change":"2024-04-25T19:02:44.157149+0000","last_active":"2024-04-25T19:02:45.200618+0000","last_peered":"2024-04-25T19:02:45.200618+0000","last_clean":"2024-04-25T19:02:45.200618+0000","last_became_active":"2024-04-25T19:02:44.156656+0000","last_became_peered":"2024-04-25T19:02:44.156656+0000","last_unstale":"2024-04-25T19:02:45.200618+0000","last_undegraded":"2024-04-25T19:02:45.200618+0000","last_fullsized":"2024-04-25T19:02:45.200618+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:14.674405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198854+0000","last_change":"2024-04-25T19:02:40.080039+0000","last_active":"2024-04-25T19:02:45.198854+0000","last_peered":"2024-04-25T19:02:45.198854+0000","last_clean":"2024-04-25T19:02:45.198854+0000","last_became_active":"2024-04-25T19:02:40.079725+0000","last_became_peered":"2024-04-25T19:02:40.079725+0000","last_unstale":"2024-04-25T19:02:45.198854+0000","last_undegraded":"2024-04-25T19:02:45.198854+0000","last_fullsized":"2024-04-25T19:02:45.198854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136108+0000","last_change":"2024-04-25T19:02:39.051841+0000","last_active":"2024-04-25T19:02:45.136108+0000","last_peered":"2024-04-25T19:02:45.136108+0000","last_clean":"2024-04-25T19:02:45.136108+0000","last_became_active":"2024-04-25T19:02:39.050695+0000","last_became_peered":"2024-04-25T19:02:39.050695+0000","last_unstale":"2024-04-25T19:02:45.136108+0000","last_undegraded":"2024-04-25T19:02:45.136108+0000","last_fullsized":"2024-04-25T19:02:45.136108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:45.632351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128669+0000","last_change":"2024-04-25T19:02:44.147067+0000","last_active":"2024-04-25T19:02:45.128669+0000","last_peered":"2024-04-25T19:02:45.128669+0000","last_clean":"2024-04-25T19:02:45.128669+0000","last_became_active":"2024-04-25T19:02:44.146048+0000","last_became_peered":"2024-04-25T19:02:44.146048+0000","last_unstale":"2024-04-25T19:02:45.128669+0000","last_undegraded":"2024-04-25T19:02:45.128669+0000","last_fullsized":"2024-04-25T19:02:45.128669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:13.605566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199716+0000","last_change":"2024-04-25T19:02:40.066529+0000","last_active":"2024-04-25T19:02:45.199716+0000","last_peered":"2024-04-25T19:02:45.199716+0000","last_clean":"2024-04-25T19:02:45.199716+0000","last_became_active":"2024-04-25T19:02:40.064814+0000","last_became_peered":"2024-04-25T19:02:40.064814+0000","last_unstale":"2024-04-25T19:02:45.199716+0000","last_undegraded":"2024-04-25T19:02:45.199716+0000","last_fullsized":"2024-04-25T19:02:45.199716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199020+0000","last_change":"2024-04-25T19:02:39.051220+0000","last_active":"2024-04-25T19:02:45.199020+0000","last_peered":"2024-04-25T19:02:45.199020+0000","last_clean":"2024-04-25T19:02:45.199020+0000","last_became_active":"2024-04-25T19:02:39.050911+0000","last_became_peered":"2024-04-25T19:02:39.050911+0000","last_unstale":"2024-04-25T19:02:45.199020+0000","last_undegraded":"2024-04-25T19:02:45.199020+0000","last_fullsized":"2024-04-25T19:02:45.199020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196897+0000","last_change":"2024-04-25T19:02:44.156822+0000","last_active":"2024-04-25T19:02:45.196897+0000","last_peered":"2024-04-25T19:02:45.196897+0000","last_clean":"2024-04-25T19:02:45.196897+0000","last_became_active":"2024-04-25T19:02:44.156379+0000","last_became_peered":"2024-04-25T19:02:44.156379+0000","last_unstale":"2024-04-25T19:02:45.196897+0000","last_undegraded":"2024-04-25T19:02:45.196897+0000","last_fullsized":"2024-04-25T19:02:45.196897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:51.422599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128832+0000","last_change":"2024-04-25T19:02:40.075607+0000","last_active":"2024-04-25T19:02:45.128832+0000","last_peered":"2024-04-25T19:02:45.128832+0000","last_clean":"2024-04-25T19:02:45.128832+0000","last_became_active":"2024-04-25T19:02:40.075267+0000","last_became_peered":"2024-04-25T19:02:40.075267+0000","last_unstale":"2024-04-25T19:02:45.128832+0000","last_undegraded":"2024-04-25T19:02:45.128832+0000","last_fullsized":"2024-04-25T19:02:45.128832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131535+0000","last_change":"2024-04-25T19:02:39.044744+0000","last_active":"2024-04-25T19:02:45.131535+0000","last_peered":"2024-04-25T19:02:45.131535+0000","last_clean":"2024-04-25T19:02:45.131535+0000","last_became_active":"2024-04-25T19:02:39.044372+0000","last_became_peered":"2024-04-25T19:02:39.044372+0000","last_unstale":"2024-04-25T19:02:45.131535+0000","last_undegraded":"2024-04-25T19:02:45.131535+0000","last_fullsized":"2024-04-25T19:02:45.131535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203015+0000","last_change":"2024-04-25T19:02:44.158146+0000","last_active":"2024-04-25T19:02:45.203015+0000","last_peered":"2024-04-25T19:02:45.203015+0000","last_clean":"2024-04-25T19:02:45.203015+0000","last_became_active":"2024-04-25T19:02:44.157017+0000","last_became_peered":"2024-04-25T19:02:44.157017+0000","last_unstale":"2024-04-25T19:02:45.203015+0000","last_undegraded":"2024-04-25T19:02:45.203015+0000","last_fullsized":"2024-04-25T19:02:45.203015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:40.012791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.196618+0000","last_change":"2024-04-25T19:02:40.073754+0000","last_active":"2024-04-25T19:02:45.196618+0000","last_peered":"2024-04-25T19:02:45.196618+0000","last_clean":"2024-04-25T19:02:45.196618+0000","last_became_active":"2024-04-25T19:02:40.073500+0000","last_became_peered":"2024-04-25T19:02:40.073500+0000","last_unstale":"2024-04-25T19:02:45.196618+0000","last_undegraded":"2024-04-25T19:02:45.196618+0000","last_fullsized":"2024-04-25T19:02:45.196618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197302+0000","last_change":"2024-04-25T19:02:39.039435+0000","last_active":"2024-04-25T19:02:45.197302+0000","last_peered":"2024-04-25T19:02:45.197302+0000","last_clean":"2024-04-25T19:02:45.197302+0000","last_became_active":"2024-04-25T19:02:39.039084+0000","last_became_peered":"2024-04-25T19:02:39.039084+0000","last_unstale":"2024-04-25T19:02:45.197302+0000","last_undegraded":"2024-04-25T19:02:45.197302+0000","last_fullsized":"2024-04-25T19:02:45.197302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202918+0000","last_change":"2024-04-25T19:02:44.156161+0000","last_active":"2024-04-25T19:02:45.202918+0000","last_peered":"2024-04-25T19:02:45.202918+0000","last_clean":"2024-04-25T19:02:45.202918+0000","last_became_active":"2024-04-25T19:02:44.155666+0000","last_became_peered":"2024-04-25T19:02:44.155666+0000","last_unstale":"2024-04-25T19:02:45.202918+0000","last_undegraded":"2024-04-25T19:02:45.202918+0000","last_fullsized":"2024-04-25T19:02:45.202918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:22:00.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129573+0000","last_change":"2024-04-25T19:02:40.076702+0000","last_active":"2024-04-25T19:02:45.129573+0000","last_peered":"2024-04-25T19:02:45.129573+0000","last_clean":"2024-04-25T19:02:45.129573+0000","last_became_active":"2024-04-25T19:02:40.072663+0000","last_became_peered":"2024-04-25T19:02:40.072663+0000","last_unstale":"2024-04-25T19:02:45.129573+0000","last_undegraded":"2024-04-25T19:02:45.129573+0000","last_fullsized":"2024-04-25T19:02:45.129573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:35.926577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130496+0000","last_change":"2024-04-25T19:02:39.048307+0000","last_active":"2024-04-25T19:02:45.130496+0000","last_peered":"2024-04-25T19:02:45.130496+0000","last_clean":"2024-04-25T19:02:45.130496+0000","last_became_active":"2024-04-25T19:02:39.047902+0000","last_became_peered":"2024-04-25T19:02:39.047902+0000","last_unstale":"2024-04-25T19:02:45.130496+0000","last_undegraded":"2024-04-25T19:02:45.130496+0000","last_fullsized":"2024-04-25T19:02:45.130496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.819804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.136907+0000","last_change":"2024-04-25T19:02:44.151849+0000","last_active":"2024-04-25T19:02:45.136907+0000","last_peered":"2024-04-25T19:02:45.136907+0000","last_clean":"2024-04-25T19:02:45.136907+0000","last_became_active":"2024-04-25T19:02:44.150781+0000","last_became_peered":"2024-04-25T19:02:44.150781+0000","last_unstale":"2024-04-25T19:02:45.136907+0000","last_undegraded":"2024-04-25T19:02:45.136907+0000","last_fullsized":"2024-04-25T19:02:45.136907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:30.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.134288+0000","last_change":"2024-04-25T19:02:40.073292+0000","last_active":"2024-04-25T19:02:45.134288+0000","last_peered":"2024-04-25T19:02:45.134288+0000","last_clean":"2024-04-25T19:02:45.134288+0000","last_became_active":"2024-04-25T19:02:40.070890+0000","last_became_peered":"2024-04-25T19:02:40.070890+0000","last_unstale":"2024-04-25T19:02:45.134288+0000","last_undegraded":"2024-04-25T19:02:45.134288+0000","last_fullsized":"2024-04-25T19:02:45.134288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:19.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201740+0000","last_change":"2024-04-25T19:02:39.053287+0000","last_active":"2024-04-25T19:02:45.201740+0000","last_peered":"2024-04-25T19:02:45.201740+0000","last_clean":"2024-04-25T19:02:45.201740+0000","last_became_active":"2024-04-25T19:02:39.052037+0000","last_became_peered":"2024-04-25T19:02:39.052037+0000","last_unstale":"2024-04-25T19:02:45.201740+0000","last_undegraded":"2024-04-25T19:02:45.201740+0000","last_fullsized":"2024-04-25T19:02:45.201740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130560+0000","last_change":"2024-04-25T19:02:44.158840+0000","last_active":"2024-04-25T19:02:45.130560+0000","last_peered":"2024-04-25T19:02:45.130560+0000","last_clean":"2024-04-25T19:02:45.130560+0000","last_became_active":"2024-04-25T19:02:44.156508+0000","last_became_peered":"2024-04-25T19:02:44.156508+0000","last_unstale":"2024-04-25T19:02:45.130560+0000","last_undegraded":"2024-04-25T19:02:45.130560+0000","last_fullsized":"2024-04-25T19:02:45.130560+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:08.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198990+0000","last_change":"2024-04-25T19:02:40.084277+0000","last_active":"2024-04-25T19:02:45.198990+0000","last_peered":"2024-04-25T19:02:45.198990+0000","last_clean":"2024-04-25T19:02:45.198990+0000","last_became_active":"2024-04-25T19:02:40.083896+0000","last_became_peered":"2024-04-25T19:02:40.083896+0000","last_unstale":"2024-04-25T19:02:45.198990+0000","last_undegraded":"2024-04-25T19:02:45.198990+0000","last_fullsized":"2024-04-25T19:02:45.198990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:31.505110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.393874+0000","last_change":"2024-04-25T19:02:39.045230+0000","last_active":"2024-04-25T19:03:00.393874+0000","last_peered":"2024-04-25T19:03:00.393874+0000","last_clean":"2024-04-25T19:03:00.393874+0000","last_became_active":"2024-04-25T19:02:39.044884+0000","last_became_peered":"2024-04-25T19:02:39.044884+0000","last_unstale":"2024-04-25T19:03:00.393874+0000","last_undegraded":"2024-04-25T19:03:00.393874+0000","last_fullsized":"2024-04-25T19:03:00.393874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:20.905809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133971+0000","last_change":"2024-04-25T19:02:44.153433+0000","last_active":"2024-04-25T19:02:45.133971+0000","last_peered":"2024-04-25T19:02:45.133971+0000","last_clean":"2024-04-25T19:02:45.133971+0000","last_became_active":"2024-04-25T19:02:44.152935+0000","last_became_peered":"2024-04-25T19:02:44.152935+0000","last_unstale":"2024-04-25T19:02:45.133971+0000","last_undegraded":"2024-04-25T19:02:45.133971+0000","last_fullsized":"2024-04-25T19:02:45.133971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:35.591324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129614+0000","last_change":"2024-04-25T19:02:40.074340+0000","last_active":"2024-04-25T19:02:45.129614+0000","last_peered":"2024-04-25T19:02:45.129614+0000","last_clean":"2024-04-25T19:02:45.129614+0000","last_became_active":"2024-04-25T19:02:40.069150+0000","last_became_peered":"2024-04-25T19:02:40.069150+0000","last_unstale":"2024-04-25T19:02:45.129614+0000","last_undegraded":"2024-04-25T19:02:45.129614+0000","last_fullsized":"2024-04-25T19:02:45.129614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:53.328375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203162+0000","last_change":"2024-04-25T19:02:39.050352+0000","last_active":"2024-04-25T19:02:45.203162+0000","last_peered":"2024-04-25T19:02:45.203162+0000","last_clean":"2024-04-25T19:02:45.203162+0000","last_became_active":"2024-04-25T19:02:39.049837+0000","last_became_peered":"2024-04-25T19:02:39.049837+0000","last_unstale":"2024-04-25T19:02:45.203162+0000","last_undegraded":"2024-04-25T19:02:45.203162+0000","last_fullsized":"2024-04-25T19:02:45.203162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:46.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200045+0000","last_change":"2024-04-25T19:02:44.147062+0000","last_active":"2024-04-25T19:02:45.200045+0000","last_peered":"2024-04-25T19:02:45.200045+0000","last_clean":"2024-04-25T19:02:45.200045+0000","last_became_active":"2024-04-25T19:02:44.145997+0000","last_became_peered":"2024-04-25T19:02:44.145997+0000","last_unstale":"2024-04-25T19:02:45.200045+0000","last_undegraded":"2024-04-25T19:02:45.200045+0000","last_fullsized":"2024-04-25T19:02:45.200045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.203397+0000","last_change":"2024-04-25T19:02:40.083756+0000","last_active":"2024-04-25T19:02:45.203397+0000","last_peered":"2024-04-25T19:02:45.203397+0000","last_clean":"2024-04-25T19:02:45.203397+0000","last_became_active":"2024-04-25T19:02:40.083402+0000","last_became_peered":"2024-04-25T19:02:40.083402+0000","last_unstale":"2024-04-25T19:02:45.203397+0000","last_undegraded":"2024-04-25T19:02:45.203397+0000","last_fullsized":"2024-04-25T19:02:45.203397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198142+0000","last_change":"2024-04-25T19:02:39.043121+0000","last_active":"2024-04-25T19:02:45.198142+0000","last_peered":"2024-04-25T19:02:45.198142+0000","last_clean":"2024-04-25T19:02:45.198142+0000","last_became_active":"2024-04-25T19:02:39.042778+0000","last_became_peered":"2024-04-25T19:02:39.042778+0000","last_unstale":"2024-04-25T19:02:45.198142+0000","last_undegraded":"2024-04-25T19:02:45.198142+0000","last_fullsized":"2024-04-25T19:02:45.198142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199456+0000","last_change":"2024-04-25T19:02:44.148628+0000","last_active":"2024-04-25T19:02:45.199456+0000","last_peered":"2024-04-25T19:02:45.199456+0000","last_clean":"2024-04-25T19:02:45.199456+0000","last_became_active":"2024-04-25T19:02:44.147929+0000","last_became_peered":"2024-04-25T19:02:44.147929+0000","last_unstale":"2024-04-25T19:02:45.199456+0000","last_undegraded":"2024-04-25T19:02:45.199456+0000","last_fullsized":"2024-04-25T19:02:45.199456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.656291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199371+0000","last_change":"2024-04-25T19:02:40.069980+0000","last_active":"2024-04-25T19:02:45.199371+0000","last_peered":"2024-04-25T19:02:45.199371+0000","last_clean":"2024-04-25T19:02:45.199371+0000","last_became_active":"2024-04-25T19:02:40.069323+0000","last_became_peered":"2024-04-25T19:02:40.069323+0000","last_unstale":"2024-04-25T19:02:45.199371+0000","last_undegraded":"2024-04-25T19:02:45.199371+0000","last_fullsized":"2024-04-25T19:02:45.199371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200662+0000","last_change":"2024-04-25T19:02:39.044027+0000","last_active":"2024-04-25T19:02:45.200662+0000","last_peered":"2024-04-25T19:02:45.200662+0000","last_clean":"2024-04-25T19:02:45.200662+0000","last_became_active":"2024-04-25T19:02:39.043696+0000","last_became_peered":"2024-04-25T19:02:39.043696+0000","last_unstale":"2024-04-25T19:02:45.200662+0000","last_undegraded":"2024-04-25T19:02:45.200662+0000","last_fullsized":"2024-04-25T19:02:45.200662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:30.086159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132192+0000","last_change":"2024-04-25T19:02:44.147513+0000","last_active":"2024-04-25T19:02:45.132192+0000","last_peered":"2024-04-25T19:02:45.132192+0000","last_clean":"2024-04-25T19:02:45.132192+0000","last_became_active":"2024-04-25T19:02:44.146937+0000","last_became_peered":"2024-04-25T19:02:44.146937+0000","last_unstale":"2024-04-25T19:02:45.132192+0000","last_undegraded":"2024-04-25T19:02:45.132192+0000","last_fullsized":"2024-04-25T19:02:45.132192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:21.888369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135996+0000","last_change":"2024-04-25T19:02:44.153340+0000","last_active":"2024-04-25T19:02:45.135996+0000","last_peered":"2024-04-25T19:02:45.135996+0000","last_clean":"2024-04-25T19:02:45.135996+0000","last_became_active":"2024-04-25T19:02:44.152850+0000","last_became_peered":"2024-04-25T19:02:44.152850+0000","last_unstale":"2024-04-25T19:02:45.135996+0000","last_undegraded":"2024-04-25T19:02:45.135996+0000","last_fullsized":"2024-04-25T19:02:45.135996+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:57.414588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132085+0000","last_change":"2024-04-25T19:02:40.051844+0000","last_active":"2024-04-25T19:02:45.132085+0000","last_peered":"2024-04-25T19:02:45.132085+0000","last_clean":"2024-04-25T19:02:45.132085+0000","last_became_active":"2024-04-25T19:02:40.050922+0000","last_became_peered":"2024-04-25T19:02:40.050922+0000","last_unstale":"2024-04-25T19:02:45.132085+0000","last_undegraded":"2024-04-25T19:02:45.132085+0000","last_fullsized":"2024-04-25T19:02:45.132085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.344370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129179+0000","last_change":"2024-04-25T19:02:39.045270+0000","last_active":"2024-04-25T19:02:45.129179+0000","last_peered":"2024-04-25T19:02:45.129179+0000","last_clean":"2024-04-25T19:02:45.129179+0000","last_became_active":"2024-04-25T19:02:39.044936+0000","last_became_peered":"2024-04-25T19:02:39.044936+0000","last_unstale":"2024-04-25T19:02:45.129179+0000","last_undegraded":"2024-04-25T19:02:45.129179+0000","last_fullsized":"2024-04-25T19:02:45.129179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:38:30.060659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131178+0000","last_change":"2024-04-25T19:02:44.160881+0000","last_active":"2024-04-25T19:02:45.131178+0000","last_peered":"2024-04-25T19:02:45.131178+0000","last_clean":"2024-04-25T19:02:45.131178+0000","last_became_active":"2024-04-25T19:02:44.160267+0000","last_became_peered":"2024-04-25T19:02:44.160267+0000","last_unstale":"2024-04-25T19:02:45.131178+0000","last_undegraded":"2024-04-25T19:02:45.131178+0000","last_fullsized":"2024-04-25T19:02:45.131178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:21.942974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199361+0000","last_change":"2024-04-25T19:02:40.065898+0000","last_active":"2024-04-25T19:02:45.199361+0000","last_peered":"2024-04-25T19:02:45.199361+0000","last_clean":"2024-04-25T19:02:45.199361+0000","last_became_active":"2024-04-25T19:02:40.062397+0000","last_became_peered":"2024-04-25T19:02:40.062397+0000","last_unstale":"2024-04-25T19:02:45.199361+0000","last_undegraded":"2024-04-25T19:02:45.199361+0000","last_fullsized":"2024-04-25T19:02:45.199361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:52.427433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132878+0000","last_change":"2024-04-25T19:02:39.049593+0000","last_active":"2024-04-25T19:02:45.132878+0000","last_peered":"2024-04-25T19:02:45.132878+0000","last_clean":"2024-04-25T19:02:45.132878+0000","last_became_active":"2024-04-25T19:02:39.049140+0000","last_became_peered":"2024-04-25T19:02:39.049140+0000","last_unstale":"2024-04-25T19:02:45.132878+0000","last_undegraded":"2024-04-25T19:02:45.132878+0000","last_fullsized":"2024-04-25T19:02:45.132878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:01.624014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130188+0000","last_change":"2024-04-25T19:02:39.050611+0000","last_active":"2024-04-25T19:02:45.130188+0000","last_peered":"2024-04-25T19:02:45.130188+0000","last_clean":"2024-04-25T19:02:45.130188+0000","last_became_active":"2024-04-25T19:02:39.050319+0000","last_became_peered":"2024-04-25T19:02:39.050319+0000","last_unstale":"2024-04-25T19:02:45.130188+0000","last_undegraded":"2024-04-25T19:02:45.130188+0000","last_fullsized":"2024-04-25T19:02:45.130188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:03.331881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132040+0000","last_change":"2024-04-25T19:02:40.076010+0000","last_active":"2024-04-25T19:02:45.132040+0000","last_peered":"2024-04-25T19:02:45.132040+0000","last_clean":"2024-04-25T19:02:45.132040+0000","last_became_active":"2024-04-25T19:02:40.075677+0000","last_became_peered":"2024-04-25T19:02:40.075677+0000","last_unstale":"2024-04-25T19:02:45.132040+0000","last_undegraded":"2024-04-25T19:02:45.132040+0000","last_fullsized":"2024-04-25T19:02:45.132040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135539+0000","last_change":"2024-04-25T19:02:44.154362+0000","last_active":"2024-04-25T19:02:45.135539+0000","last_peered":"2024-04-25T19:02:45.135539+0000","last_clean":"2024-04-25T19:02:45.135539+0000","last_became_active":"2024-04-25T19:02:44.153509+0000","last_became_peered":"2024-04-25T19:02:44.153509+0000","last_unstale":"2024-04-25T19:02:45.135539+0000","last_undegraded":"2024-04-25T19:02:45.135539+0000","last_fullsized":"2024-04-25T19:02:45.135539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:16.467033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201054+0000","last_change":"2024-04-25T19:02:39.045741+0000","last_active":"2024-04-25T19:02:45.201054+0000","last_peered":"2024-04-25T19:02:45.201054+0000","last_clean":"2024-04-25T19:02:45.201054+0000","last_became_active":"2024-04-25T19:02:39.045015+0000","last_became_peered":"2024-04-25T19:02:39.045015+0000","last_unstale":"2024-04-25T19:02:45.201054+0000","last_undegraded":"2024-04-25T19:02:45.201054+0000","last_fullsized":"2024-04-25T19:02:45.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:37.993475+0000","last_clean_scrub_stamp":"2024-04-25T19:02:37.993475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:12.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199483+0000","last_change":"2024-04-25T19:02:40.080804+0000","last_active":"2024-04-25T19:02:45.199483+0000","last_peered":"2024-04-25T19:02:45.199483+0000","last_clean":"2024-04-25T19:02:45.199483+0000","last_became_active":"2024-04-25T19:02:40.080421+0000","last_became_peered":"2024-04-25T19:02:40.080421+0000","last_unstale":"2024-04-25T19:02:45.199483+0000","last_undegraded":"2024-04-25T19:02:45.199483+0000","last_fullsized":"2024-04-25T19:02:45.199483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:35:18.628087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202500+0000","last_change":"2024-04-25T19:02:44.163067+0000","last_active":"2024-04-25T19:02:45.202500+0000","last_peered":"2024-04-25T19:02:45.202500+0000","last_clean":"2024-04-25T19:02:45.202500+0000","last_became_active":"2024-04-25T19:02:44.162000+0000","last_became_peered":"2024-04-25T19:02:44.162000+0000","last_unstale":"2024-04-25T19:02:45.202500+0000","last_undegraded":"2024-04-25T19:02:45.202500+0000","last_fullsized":"2024-04-25T19:02:45.202500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:43.085171+0000","last_clean_scrub_stamp":"2024-04-25T19:02:43.085171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:59.430583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198599+0000","last_change":"2024-04-25T19:02:40.063088+0000","last_active":"2024-04-25T19:02:45.198599+0000","last_peered":"2024-04-25T19:02:45.198599+0000","last_clean":"2024-04-25T19:02:45.198599+0000","last_became_active":"2024-04-25T19:02:40.062379+0000","last_became_peered":"2024-04-25T19:02:40.062379+0000","last_unstale":"2024-04-25T19:02:45.198599+0000","last_undegraded":"2024-04-25T19:02:45.198599+0000","last_fullsized":"2024-04-25T19:02:45.198599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:13.825286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200902+0000","last_change":"2024-04-25T19:02:40.065240+0000","last_active":"2024-04-25T19:02:45.200902+0000","last_peered":"2024-04-25T19:02:45.200902+0000","last_clean":"2024-04-25T19:02:45.200902+0000","last_became_active":"2024-04-25T19:02:40.064900+0000","last_became_peered":"2024-04-25T19:02:40.064900+0000","last_unstale":"2024-04-25T19:02:45.200902+0000","last_undegraded":"2024-04-25T19:02:45.200902+0000","last_fullsized":"2024-04-25T19:02:45.200902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:34.156431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.129604+0000","last_change":"2024-04-25T19:02:40.084316+0000","last_active":"2024-04-25T19:02:45.129604+0000","last_peered":"2024-04-25T19:02:45.129604+0000","last_clean":"2024-04-25T19:02:45.129604+0000","last_became_active":"2024-04-25T19:02:40.084014+0000","last_became_peered":"2024-04-25T19:02:40.084014+0000","last_unstale":"2024-04-25T19:02:45.129604+0000","last_undegraded":"2024-04-25T19:02:45.129604+0000","last_fullsized":"2024-04-25T19:02:45.129604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:12.380820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200778+0000","last_change":"2024-04-25T19:02:40.070357+0000","last_active":"2024-04-25T19:02:45.200778+0000","last_peered":"2024-04-25T19:02:45.200778+0000","last_clean":"2024-04-25T19:02:45.200778+0000","last_became_active":"2024-04-25T19:02:40.070028+0000","last_became_peered":"2024-04-25T19:02:40.070028+0000","last_unstale":"2024-04-25T19:02:45.200778+0000","last_undegraded":"2024-04-25T19:02:45.200778+0000","last_fullsized":"2024-04-25T19:02:45.200778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137153+0000","last_change":"2024-04-25T19:02:40.088098+0000","last_active":"2024-04-25T19:02:45.137153+0000","last_peered":"2024-04-25T19:02:45.137153+0000","last_clean":"2024-04-25T19:02:45.137153+0000","last_became_active":"2024-04-25T19:02:40.087738+0000","last_became_peered":"2024-04-25T19:02:40.087738+0000","last_unstale":"2024-04-25T19:02:45.137153+0000","last_undegraded":"2024-04-25T19:02:45.137153+0000","last_fullsized":"2024-04-25T19:02:45.137153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:08.428103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.137734+0000","last_change":"2024-04-25T19:02:40.063201+0000","last_active":"2024-04-25T19:02:45.137734+0000","last_peered":"2024-04-25T19:02:45.137734+0000","last_clean":"2024-04-25T19:02:45.137734+0000","last_became_active":"2024-04-25T19:02:40.060739+0000","last_became_peered":"2024-04-25T19:02:40.060739+0000","last_unstale":"2024-04-25T19:02:45.137734+0000","last_undegraded":"2024-04-25T19:02:45.137734+0000","last_fullsized":"2024-04-25T19:02:45.137734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:40.756295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199236+0000","last_change":"2024-04-25T19:02:40.076099+0000","last_active":"2024-04-25T19:02:45.199236+0000","last_peered":"2024-04-25T19:02:45.199236+0000","last_clean":"2024-04-25T19:02:45.199236+0000","last_became_active":"2024-04-25T19:02:40.075690+0000","last_became_peered":"2024-04-25T19:02:40.075690+0000","last_unstale":"2024-04-25T19:02:45.199236+0000","last_undegraded":"2024-04-25T19:02:45.199236+0000","last_fullsized":"2024-04-25T19:02:45.199236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:51.312420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202228+0000","last_change":"2024-04-25T19:02:40.075377+0000","last_active":"2024-04-25T19:02:45.202228+0000","last_peered":"2024-04-25T19:02:45.202228+0000","last_clean":"2024-04-25T19:02:45.202228+0000","last_became_active":"2024-04-25T19:02:40.069488+0000","last_became_peered":"2024-04-25T19:02:40.069488+0000","last_unstale":"2024-04-25T19:02:45.202228+0000","last_undegraded":"2024-04-25T19:02:45.202228+0000","last_fullsized":"2024-04-25T19:02:45.202228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:48.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128848+0000","last_change":"2024-04-25T19:02:40.083952+0000","last_active":"2024-04-25T19:02:45.128848+0000","last_peered":"2024-04-25T19:02:45.128848+0000","last_clean":"2024-04-25T19:02:45.128848+0000","last_became_active":"2024-04-25T19:02:40.083612+0000","last_became_peered":"2024-04-25T19:02:40.083612+0000","last_unstale":"2024-04-25T19:02:45.128848+0000","last_undegraded":"2024-04-25T19:02:45.128848+0000","last_fullsized":"2024-04-25T19:02:45.128848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.199584+0000","last_change":"2024-04-25T19:02:40.067612+0000","last_active":"2024-04-25T19:02:45.199584+0000","last_peered":"2024-04-25T19:02:45.199584+0000","last_clean":"2024-04-25T19:02:45.199584+0000","last_became_active":"2024-04-25T19:02:40.067294+0000","last_became_peered":"2024-04-25T19:02:40.067294+0000","last_unstale":"2024-04-25T19:02:45.199584+0000","last_undegraded":"2024-04-25T19:02:45.199584+0000","last_fullsized":"2024-04-25T19:02:45.199584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197462+0000","last_change":"2024-04-25T19:02:40.073442+0000","last_active":"2024-04-25T19:02:45.197462+0000","last_peered":"2024-04-25T19:02:45.197462+0000","last_clean":"2024-04-25T19:02:45.197462+0000","last_became_active":"2024-04-25T19:02:40.073045+0000","last_became_peered":"2024-04-25T19:02:40.073045+0000","last_unstale":"2024-04-25T19:02:45.197462+0000","last_undegraded":"2024-04-25T19:02:45.197462+0000","last_fullsized":"2024-04-25T19:02:45.197462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:42.466361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.131761+0000","last_change":"2024-04-25T19:02:40.050494+0000","last_active":"2024-04-25T19:02:45.131761+0000","last_peered":"2024-04-25T19:02:45.131761+0000","last_clean":"2024-04-25T19:02:45.131761+0000","last_became_active":"2024-04-25T19:02:40.050067+0000","last_became_peered":"2024-04-25T19:02:40.050067+0000","last_unstale":"2024-04-25T19:02:45.131761+0000","last_undegraded":"2024-04-25T19:02:45.131761+0000","last_fullsized":"2024-04-25T19:02:45.131761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198872+0000","last_change":"2024-04-25T19:02:40.071347+0000","last_active":"2024-04-25T19:02:45.198872+0000","last_peered":"2024-04-25T19:02:45.198872+0000","last_clean":"2024-04-25T19:02:45.198872+0000","last_became_active":"2024-04-25T19:02:40.071062+0000","last_became_peered":"2024-04-25T19:02:40.071062+0000","last_unstale":"2024-04-25T19:02:45.198872+0000","last_undegraded":"2024-04-25T19:02:45.198872+0000","last_fullsized":"2024-04-25T19:02:45.198872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.130821+0000","last_change":"2024-04-25T19:02:40.076710+0000","last_active":"2024-04-25T19:02:45.130821+0000","last_peered":"2024-04-25T19:02:45.130821+0000","last_clean":"2024-04-25T19:02:45.130821+0000","last_became_active":"2024-04-25T19:02:40.071132+0000","last_became_peered":"2024-04-25T19:02:40.071132+0000","last_unstale":"2024-04-25T19:02:45.130821+0000","last_undegraded":"2024-04-25T19:02:45.130821+0000","last_fullsized":"2024-04-25T19:02:45.130821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:21:56.124743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202755+0000","last_change":"2024-04-25T19:02:40.076746+0000","last_active":"2024-04-25T19:02:45.202755+0000","last_peered":"2024-04-25T19:02:45.202755+0000","last_clean":"2024-04-25T19:02:45.202755+0000","last_became_active":"2024-04-25T19:02:40.068835+0000","last_became_peered":"2024-04-25T19:02:40.068835+0000","last_unstale":"2024-04-25T19:02:45.202755+0000","last_undegraded":"2024-04-25T19:02:45.202755+0000","last_fullsized":"2024-04-25T19:02:45.202755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.132119+0000","last_change":"2024-04-25T19:02:40.090029+0000","last_active":"2024-04-25T19:02:45.132119+0000","last_peered":"2024-04-25T19:02:45.132119+0000","last_clean":"2024-04-25T19:02:45.132119+0000","last_became_active":"2024-04-25T19:02:40.088886+0000","last_became_peered":"2024-04-25T19:02:40.088886+0000","last_unstale":"2024-04-25T19:02:45.132119+0000","last_undegraded":"2024-04-25T19:02:45.132119+0000","last_fullsized":"2024-04-25T19:02:45.132119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:38.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.128770+0000","last_change":"2024-04-25T19:02:40.079792+0000","last_active":"2024-04-25T19:02:45.128770+0000","last_peered":"2024-04-25T19:02:45.128770+0000","last_clean":"2024-04-25T19:02:45.128770+0000","last_became_active":"2024-04-25T19:02:40.071455+0000","last_became_peered":"2024-04-25T19:02:40.071455+0000","last_unstale":"2024-04-25T19:02:45.128770+0000","last_undegraded":"2024-04-25T19:02:45.128770+0000","last_fullsized":"2024-04-25T19:02:45.128770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198995+0000","last_change":"2024-04-25T19:02:40.088240+0000","last_active":"2024-04-25T19:02:45.198995+0000","last_peered":"2024-04-25T19:02:45.198995+0000","last_clean":"2024-04-25T19:02:45.198995+0000","last_became_active":"2024-04-25T19:02:40.087892+0000","last_became_peered":"2024-04-25T19:02:40.087892+0000","last_unstale":"2024-04-25T19:02:45.198995+0000","last_undegraded":"2024-04-25T19:02:45.198995+0000","last_fullsized":"2024-04-25T19:02:45.198995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:04.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.133970+0000","last_change":"2024-04-25T19:02:40.062612+0000","last_active":"2024-04-25T19:02:45.133970+0000","last_peered":"2024-04-25T19:02:45.133970+0000","last_clean":"2024-04-25T19:02:45.133970+0000","last_became_active":"2024-04-25T19:02:40.060475+0000","last_became_peered":"2024-04-25T19:02:40.060475+0000","last_unstale":"2024-04-25T19:02:45.133970+0000","last_undegraded":"2024-04-25T19:02:45.133970+0000","last_fullsized":"2024-04-25T19:02:45.133970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:10.198621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.135782+0000","last_change":"2024-04-25T19:02:40.091050+0000","last_active":"2024-04-25T19:02:45.135782+0000","last_peered":"2024-04-25T19:02:45.135782+0000","last_clean":"2024-04-25T19:02:45.135782+0000","last_became_active":"2024-04-25T19:02:40.090640+0000","last_became_peered":"2024-04-25T19:02:40.090640+0000","last_unstale":"2024-04-25T19:02:45.135782+0000","last_undegraded":"2024-04-25T19:02:45.135782+0000","last_fullsized":"2024-04-25T19:02:45.135782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:57.435299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198750+0000","last_change":"2024-04-25T19:02:40.074894+0000","last_active":"2024-04-25T19:02:45.198750+0000","last_peered":"2024-04-25T19:02:45.198750+0000","last_clean":"2024-04-25T19:02:45.198750+0000","last_became_active":"2024-04-25T19:02:40.074142+0000","last_became_peered":"2024-04-25T19:02:40.074142+0000","last_unstale":"2024-04-25T19:02:45.198750+0000","last_undegraded":"2024-04-25T19:02:45.198750+0000","last_fullsized":"2024-04-25T19:02:45.198750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:17.802156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198798+0000","last_change":"2024-04-25T19:02:40.087831+0000","last_active":"2024-04-25T19:02:45.198798+0000","last_peered":"2024-04-25T19:02:45.198798+0000","last_clean":"2024-04-25T19:02:45.198798+0000","last_became_active":"2024-04-25T19:02:40.087495+0000","last_became_peered":"2024-04-25T19:02:40.087495+0000","last_unstale":"2024-04-25T19:02:45.198798+0000","last_undegraded":"2024-04-25T19:02:45.198798+0000","last_fullsized":"2024-04-25T19:02:45.198798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.908520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197993+0000","last_change":"2024-04-25T19:02:40.067092+0000","last_active":"2024-04-25T19:02:45.197993+0000","last_peered":"2024-04-25T19:02:45.197993+0000","last_clean":"2024-04-25T19:02:45.197993+0000","last_became_active":"2024-04-25T19:02:40.066752+0000","last_became_peered":"2024-04-25T19:02:40.066752+0000","last_unstale":"2024-04-25T19:02:45.197993+0000","last_undegraded":"2024-04-25T19:02:45.197993+0000","last_fullsized":"2024-04-25T19:02:45.197993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:58.965898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.200560+0000","last_change":"2024-04-25T19:02:40.069596+0000","last_active":"2024-04-25T19:02:45.200560+0000","last_peered":"2024-04-25T19:02:45.200560+0000","last_clean":"2024-04-25T19:02:45.200560+0000","last_became_active":"2024-04-25T19:02:40.069272+0000","last_became_peered":"2024-04-25T19:02:40.069272+0000","last_unstale":"2024-04-25T19:02:45.200560+0000","last_undegraded":"2024-04-25T19:02:45.200560+0000","last_fullsized":"2024-04-25T19:02:45.200560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:34.233447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198925+0000","last_change":"2024-04-25T19:02:40.085542+0000","last_active":"2024-04-25T19:02:45.198925+0000","last_peered":"2024-04-25T19:02:45.198925+0000","last_clean":"2024-04-25T19:02:45.198925+0000","last_became_active":"2024-04-25T19:02:40.085203+0000","last_became_peered":"2024-04-25T19:02:40.085203+0000","last_unstale":"2024-04-25T19:02:45.198925+0000","last_undegraded":"2024-04-25T19:02:45.198925+0000","last_fullsized":"2024-04-25T19:02:45.198925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:47.336386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.197180+0000","last_change":"2024-04-25T19:02:40.070062+0000","last_active":"2024-04-25T19:02:45.197180+0000","last_peered":"2024-04-25T19:02:45.197180+0000","last_clean":"2024-04-25T19:02:45.197180+0000","last_became_active":"2024-04-25T19:02:40.069224+0000","last_became_peered":"2024-04-25T19:02:40.069224+0000","last_unstale":"2024-04-25T19:02:45.197180+0000","last_undegraded":"2024-04-25T19:02:45.197180+0000","last_fullsized":"2024-04-25T19:02:45.197180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.216439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198254+0000","last_change":"2024-04-25T19:02:40.055714+0000","last_active":"2024-04-25T19:02:45.198254+0000","last_peered":"2024-04-25T19:02:45.198254+0000","last_clean":"2024-04-25T19:02:45.198254+0000","last_became_active":"2024-04-25T19:02:40.052418+0000","last_became_peered":"2024-04-25T19:02:40.052418+0000","last_unstale":"2024-04-25T19:02:45.198254+0000","last_undegraded":"2024-04-25T19:02:45.198254+0000","last_fullsized":"2024-04-25T19:02:45.198254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:04.337364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198608+0000","last_change":"2024-04-25T19:02:40.087716+0000","last_active":"2024-04-25T19:02:45.198608+0000","last_peered":"2024-04-25T19:02:45.198608+0000","last_clean":"2024-04-25T19:02:45.198608+0000","last_became_active":"2024-04-25T19:02:40.085461+0000","last_became_peered":"2024-04-25T19:02:40.085461+0000","last_unstale":"2024-04-25T19:02:45.198608+0000","last_undegraded":"2024-04-25T19:02:45.198608+0000","last_fullsized":"2024-04-25T19:02:45.198608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:21.016377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198153+0000","last_change":"2024-04-25T19:02:40.053676+0000","last_active":"2024-04-25T19:02:45.198153+0000","last_peered":"2024-04-25T19:02:45.198153+0000","last_clean":"2024-04-25T19:02:45.198153+0000","last_became_active":"2024-04-25T19:02:40.052518+0000","last_became_peered":"2024-04-25T19:02:40.052518+0000","last_unstale":"2024-04-25T19:02:45.198153+0000","last_undegraded":"2024-04-25T19:02:45.198153+0000","last_fullsized":"2024-04-25T19:02:45.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:05:54.192825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.201739+0000","last_change":"2024-04-25T19:02:40.076362+0000","last_active":"2024-04-25T19:02:45.201739+0000","last_peered":"2024-04-25T19:02:45.201739+0000","last_clean":"2024-04-25T19:02:45.201739+0000","last_became_active":"2024-04-25T19:02:40.070187+0000","last_became_peered":"2024-04-25T19:02:40.070187+0000","last_unstale":"2024-04-25T19:02:45.201739+0000","last_undegraded":"2024-04-25T19:02:45.201739+0000","last_fullsized":"2024-04-25T19:02:45.201739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:00.680292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.202327+0000","last_change":"2024-04-25T19:02:40.072359+0000","last_active":"2024-04-25T19:02:45.202327+0000","last_peered":"2024-04-25T19:02:45.202327+0000","last_clean":"2024-04-25T19:02:45.202327+0000","last_became_active":"2024-04-25T19:02:40.071678+0000","last_became_peered":"2024-04-25T19:02:40.071678+0000","last_unstale":"2024-04-25T19:02:45.202327+0000","last_undegraded":"2024-04-25T19:02:45.202327+0000","last_fullsized":"2024-04-25T19:02:45.202327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T19:02:45.198720+0000","last_change":"2024-04-25T19:02:40.078568+0000","last_active":"2024-04-25T19:02:45.198720+0000","last_peered":"2024-04-25T19:02:45.198720+0000","last_clean":"2024-04-25T19:02:45.198720+0000","last_became_active":"2024-04-25T19:02:40.078287+0000","last_became_peered":"2024-04-25T19:02:40.078287+0000","last_unstale":"2024-04-25T19:02:45.198720+0000","last_undegraded":"2024-04-25T19:02:45.198720+0000","last_fullsized":"2024-04-25T19:02:45.198720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:38.998958+0000","last_clean_scrub_stamp":"2024-04-25T19:02:38.998958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:51.767867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:11.746 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T19:03:11.746 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T19:03:11.746 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T19:03:11.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:03:12.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.052+0000 7f1293e00640 1 Processor -- start 2024-04-25T19:03:12.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.052+0000 7f1293e00640 1 -- start start 2024-04-25T19:03:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.054+0000 7f1293e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1294167e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.054+0000 7f1293e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129406f130 con 0x7f12941608b0 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.054+0000 7f1293e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129415ac20 con 0x7f129415a810 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.054+0000 7f1293e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f12940b7060 con 0x7f12941659e0 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1292000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f12941608b0 0x7f1294160cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45342/0 (socket says 172.21.15.123:45342) 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1292000640 1 -- 172.21.15.123:0/3543308722 learned_addr learned my addr 172.21.15.123:0/3543308722 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1293400640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1294167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319200585 0 0) 0x7f129406f130 con 0x7f12941608b0 2024-04-25T19:03:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1278003520 con 0x7f12941608b0 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 909079369 0 0) 0x7f1278003520 con 0x7f12941608b0 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 msgr2=0x7f1294167e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1294167e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 >> v1:172.21.15.146:6789/0 conn(0x7f129415a810 legacy=0x7f12940b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12940b7340 con 0x7f12941608b0 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f1284001690 con 0x7f12941608b0 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.055+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 >> v1:172.21.15.123:6789/0 conn(0x7f12941608b0 legacy=0x7f1294160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12780039e0 0x7f1278003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1278004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1278005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12940b7060 con 0x7f1278004d40 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f127800bbb0 con 0x7f12780039e0 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f127800bdb0 con 0x7f12941659e0 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1278005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1292000640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1278004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1292a00640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12780039e0 0x7f1278003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 -- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 msgr2=0x7f1278004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12941659e0 0x7f1278004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 -- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12780039e0 msgr2=0x7f1278003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12780039e0 0x7f1278003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1293400640 1 -- 172.21.15.123:0/3543308722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1278003520 con 0x7f1278004d40 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.056+0000 7f1292a00640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12780039e0 0x7f1278003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:12.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.057+0000 7f1293400640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1278005150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fc0 tx=0x7f1278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64d7e705a1ce635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c034020 con 0x7f1278004d40 2024-04-25T19:03:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c0305f0 con 0x7f1278004d40 2024-04-25T19:03:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1291600640 1 -- 172.21.15.123:0/3543308722 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c009c10 con 0x7f1278004d40 2024-04-25T19:03:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1293e00640 1 -- 172.21.15.123:0/3543308722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 msgr2=0x7f1278005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1293e00640 1 --2- 172.21.15.123:0/3543308722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1278005150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fc0 tx=0x7f1278005830 comp rx=0 tx=0).stop 2024-04-25T19:03:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.058+0000 7f1293400640 1 -- 172.21.15.123:0/3543308722 reap_dead start 2024-04-25T19:03:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.059+0000 7f1293e00640 1 -- 172.21.15.123:0/3543308722 shutdown_connections 2024-04-25T19:03:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.059+0000 7f1293e00640 1 -- 172.21.15.123:0/3543308722 >> 172.21.15.123:0/3543308722 conn(0x7f12940ce090 msgr2=0x7f12940ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.059+0000 7f1293e00640 1 -- 172.21.15.123:0/3543308722 shutdown_connections 2024-04-25T19:03:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.059+0000 7f1293e00640 1 -- 172.21.15.123:0/3543308722 wait complete. 2024-04-25T19:03:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.061+0000 7f1293e00640 1 Processor -- start 2024-04-25T19:03:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.061+0000 7f1293e00640 1 -- start start 2024-04-25T19:03:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1294153a50 0x7f1294155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1294155cc0 0x7f129415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12940cf9d0 con 0x7f1278004d40 2024-04-25T19:03:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f12940cf6f0 con 0x7f1294153a50 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1293e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f12940d0f00 con 0x7f1294155cc0 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1292a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1292a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34540/0 (socket says 172.21.15.123:34540) 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.064+0000 7f1292a00640 1 -- 172.21.15.123:0/3126773895 learned_addr learned my addr 172.21.15.123:0/3126773895 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1293400640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1294155cc0 0x7f129415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 -- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1294155cc0 msgr2=0x7f129415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1294155cc0 0x7f129415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 -- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1294153a50 msgr2=0x7f1294155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1294153a50 0x7f1294155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 -- 172.21.15.123:0/3126773895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12941560d0 con 0x7f1278004d40 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.065+0000 7f1292a00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d7d0 tx=0x7f127c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.066+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c005070 con 0x7f1278004d40 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.066+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c005230 con 0x7f1278004d40 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.066+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c007030 con 0x7f1278004d40 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.066+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129415c840 con 0x7f1278004d40 2024-04-25T19:03:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.066+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129415ce00 con 0x7f1278004d40 2024-04-25T19:03:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.067+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1294050740 con 0x7f1278004d40 2024-04-25T19:03:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.068+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f127c01a020 con 0x7f1278004d40 2024-04-25T19:03:12.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.069+0000 7f128aa00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 0x7f12700768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.070+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f127c094e90 con 0x7f1278004d40 2024-04-25T19:03:12.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.070+0000 7f1292000640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 0x7f12700768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.071+0000 7f1292000640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 0x7f12700768a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1284001220 tx=0x7f1284001d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.075+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12941560d0 con 0x7f1278004d40 2024-04-25T19:03:12.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.316+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f129415e3f0 con 0x7f1278004d40 2024-04-25T19:03:12.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.317+0000 7f128aa00640 1 -- 172.21.15.123:0/3126773895 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f127c096080 con 0x7f1278004d40 2024-04-25T19:03:12.319 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:12.319 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 msgr2=0x7f12700768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 0x7f12700768a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1284001220 tx=0x7f1284001d90 comp rx=0 tx=0).stop 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 msgr2=0x7f1294156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d7d0 tx=0x7f127c00dc90 comp rx=0 tx=0).stop 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 shutdown_connections 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1270074390 0x7f12700768a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1294155cc0 0x7f129415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1294153a50 0x7f1294155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 --2- 172.21.15.123:0/3126773895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278004d40 0x7f1294156680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 >> 172.21.15.123:0/3126773895 conn(0x7f12940ce090 msgr2=0x7f129406fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:12.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 shutdown_connections 2024-04-25T19:03:12.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.320+0000 7f1293e00640 1 -- 172.21.15.123:0/3126773895 wait complete. 2024-04-25T19:03:12.347 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T19:03:12.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:12.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.616+0000 7f599a800640 1 Processor -- start 2024-04-25T19:03:12.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.616+0000 7f599a800640 1 -- start start 2024-04-25T19:03:12.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.618+0000 7f599a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f599406d580 0x7f599406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.618+0000 7f599a800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599406ded0 con 0x7f599415e2f0 2024-04-25T19:03:12.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.618+0000 7f599a800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599415f070 con 0x7f599406fe20 2024-04-25T19:03:12.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.618+0000 7f599a800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f599415ee10 con 0x7f599406d580 2024-04-25T19:03:12.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5999e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f599415e2f0 0x7f599415e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45344/0 (socket says 172.21.15.123:45344) 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5999e00640 1 -- 172.21.15.123:0/81841730 learned_addr learned my addr 172.21.15.123:0/81841730 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5998a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f599406d580 0x7f599406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5998a00640 1 -- 172.21.15.123:0/81841730 >> v1:172.21.15.146:6789/0 conn(0x7f599406fe20 legacy=0x7f599415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5998a00640 1 -- 172.21.15.123:0/81841730 >> v1:172.21.15.123:6789/0 conn(0x7f599415e2f0 legacy=0x7f599415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5998a00640 1 -- 172.21.15.123:0/81841730 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599406f340 con 0x7f599406d580 2024-04-25T19:03:12.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.619+0000 7f5993e00640 1 -- 172.21.15.123:0/81841730 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658534662 0 0) 0x7f598c001490 con 0x7f599415e2f0 2024-04-25T19:03:12.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.620+0000 7f5998a00640 1 --2- 172.21.15.123:0/81841730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599406d580 0x7f599406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59800097a0 tx=0x7f598002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dd0a41d8252698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.621+0000 7f5993e00640 1 -- 172.21.15.123:0/81841730 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980037070 con 0x7f599406d580 2024-04-25T19:03:12.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.621+0000 7f5993e00640 1 -- 172.21.15.123:0/81841730 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598002ae50 con 0x7f599406d580 2024-04-25T19:03:12.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.621+0000 7f5993e00640 1 -- 172.21.15.123:0/81841730 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980032660 con 0x7f599406d580 2024-04-25T19:03:12.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.621+0000 7f599a800640 1 -- 172.21.15.123:0/81841730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599406d580 msgr2=0x7f599406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.621+0000 7f599a800640 1 --2- 172.21.15.123:0/81841730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599406d580 0x7f599406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59800097a0 tx=0x7f598002a140 comp rx=0 tx=0).stop 2024-04-25T19:03:12.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.622+0000 7f599a800640 1 -- 172.21.15.123:0/81841730 shutdown_connections 2024-04-25T19:03:12.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.622+0000 7f599a800640 1 --2- 172.21.15.123:0/81841730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599406d580 0x7f599406d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.622+0000 7f599a800640 1 -- 172.21.15.123:0/81841730 >> 172.21.15.123:0/81841730 conn(0x7f5994073200 msgr2=0x7f5994073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:12.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.622+0000 7f599a800640 1 -- 172.21.15.123:0/81841730 shutdown_connections 2024-04-25T19:03:12.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.622+0000 7f599a800640 1 -- 172.21.15.123:0/81841730 wait complete. 2024-04-25T19:03:12.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.624+0000 7f599a800640 1 Processor -- start 2024-04-25T19:03:12.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.624+0000 7f599a800640 1 -- start start 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.625+0000 7f599a800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f599406d580 0x7f599413bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.625+0000 7f599a800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f599406fe20 0x7f59940b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.625+0000 7f599a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.625+0000 7f599a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5994168c70 con 0x7f599415e2f0 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.625+0000 7f599a800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f599406f900 con 0x7f599406fe20 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f599a800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5994168df0 con 0x7f599406d580 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34562/0 (socket says 172.21.15.123:34562) 2024-04-25T19:03:12.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 -- 172.21.15.123:0/2597586988 learned_addr learned my addr 172.21.15.123:0/2597586988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5998a00640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f599406fe20 0x7f59940b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 -- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f599406d580 msgr2=0x7f599413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f599406d580 0x7f599413bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 -- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f599406fe20 msgr2=0x7f59940b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f599406fe20 0x7f59940b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.626+0000 7f5999e00640 1 -- 172.21.15.123:0/2597586988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5994166f30 con 0x7f599415e2f0 2024-04-25T19:03:12.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f5999e00640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c700 tx=0x7f598c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c018070 con 0x7f599415e2f0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598c00ce10 con 0x7f599415e2f0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c00a880 con 0x7f599415e2f0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5994167210 con 0x7f599415e2f0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.627+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59941677d0 con 0x7f599415e2f0 2024-04-25T19:03:12.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.628+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599415a6e0 con 0x7f599415e2f0 2024-04-25T19:03:12.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.629+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f598c00aa20 con 0x7f599415e2f0 2024-04-25T19:03:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.630+0000 7f5991600640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 0x7f59740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.630+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f598c094a80 con 0x7f599415e2f0 2024-04-25T19:03:12.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.630+0000 7f5999400640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 0x7f59740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:12.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.631+0000 7f5999400640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 0x7f59740769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5988006fd0 tx=0x7f5988006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:12.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.635+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f598c096e20 con 0x7f599415e2f0 2024-04-25T19:03:12.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.834+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5994070a00 con 0x7f599415e2f0 2024-04-25T19:03:12.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.837+0000 7f5991600640 1 -- 172.21.15.123:0/2597586988 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f599406ded0 con 0x7f599415e2f0 2024-04-25T19:03:12.839 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:12.839 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 msgr2=0x7f59740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 0x7f59740769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5988006fd0 tx=0x7f5988006bc0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 msgr2=0x7f59940b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c700 tx=0x7f598c00cbc0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 shutdown_connections 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59740744b0 0x7f59740769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599415e2f0 0x7f59940b6030 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f599406fe20 0x7f59940b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 --2- 172.21.15.123:0/2597586988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f599406d580 0x7f599413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:12.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 >> 172.21.15.123:0/2597586988 conn(0x7f5994073200 msgr2=0x7f599415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:12.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 shutdown_connections 2024-04-25T19:03:12.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:12.841+0000 7f599a800640 1 -- 172.21.15.123:0/2597586988 wait complete. 2024-04-25T19:03:12.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:12.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.142+0000 7f61fa800640 1 Processor -- start 2024-04-25T19:03:13.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.142+0000 7f61fa800640 1 -- start start 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61fa800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61f40b0ee0 0x7f61f4160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61fa800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f4160740 con 0x7f61f40ce420 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61fa800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f40d1a90 con 0x7f61f40cef40 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61fa800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f61f40d26a0 con 0x7f61f40b0ee0 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61f9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61f40b0ee0 0x7f61f4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.144+0000 7f61f9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61f40b0ee0 0x7f61f4160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34570/0 (socket says 172.21.15.123:34570) 2024-04-25T19:03:13.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.145+0000 7f61f9400640 1 -- 172.21.15.123:0/987362731 learned_addr learned my addr 172.21.15.123:0/987362731 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:13.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.145+0000 7f61f9400640 1 -- 172.21.15.123:0/987362731 >> v1:172.21.15.146:6789/0 conn(0x7f61f40cef40 legacy=0x7f61f40cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.145+0000 7f61f9400640 1 -- 172.21.15.123:0/987362731 >> v1:172.21.15.123:6789/0 conn(0x7f61f40ce420 legacy=0x7f61f40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.145+0000 7f61f9400640 1 -- 172.21.15.123:0/987362731 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f40d1d70 con 0x7f61f40b0ee0 2024-04-25T19:03:13.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.146+0000 7f61f9400640 1 --2- 172.21.15.123:0/987362731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40b0ee0 0x7f61f4160200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009870 tx=0x7f61e802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8dd996235b0106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.146+0000 7f61f3e00640 1 -- 172.21.15.123:0/987362731 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8037070 con 0x7f61f40b0ee0 2024-04-25T19:03:13.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.146+0000 7f61f3e00640 1 -- 172.21.15.123:0/987362731 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e8032440 con 0x7f61f40b0ee0 2024-04-25T19:03:13.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.146+0000 7f61f3e00640 1 -- 172.21.15.123:0/987362731 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8032810 con 0x7f61f40b0ee0 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 -- 172.21.15.123:0/987362731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40b0ee0 msgr2=0x7f61f4160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 --2- 172.21.15.123:0/987362731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40b0ee0 0x7f61f4160200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009870 tx=0x7f61e802a2a0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 -- 172.21.15.123:0/987362731 shutdown_connections 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 --2- 172.21.15.123:0/987362731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40b0ee0 0x7f61f4160200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 -- 172.21.15.123:0/987362731 >> 172.21.15.123:0/987362731 conn(0x7f61f409b4a0 msgr2=0x7f61f40b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 -- 172.21.15.123:0/987362731 shutdown_connections 2024-04-25T19:03:13.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.147+0000 7f61fa800640 1 -- 172.21.15.123:0/987362731 wait complete. 2024-04-25T19:03:13.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.150+0000 7f61fa800640 1 Processor -- start 2024-04-25T19:03:13.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.150+0000 7f61fa800640 1 -- start start 2024-04-25T19:03:13.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.151+0000 7f61fa800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61f40b0ee0 0x7f61f415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.151+0000 7f61fa800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61f40ce420 0x7f61f40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61fa800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61fa800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40b12f0 con 0x7f61f40cef40 2024-04-25T19:03:13.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61fa800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40d2330 con 0x7f61f40b0ee0 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61fa800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f61f40d2b30 con 0x7f61f40ce420 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34578/0 (socket says 172.21.15.123:34578) 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 -- 172.21.15.123:0/410997590 learned_addr learned my addr 172.21.15.123:0/410997590 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9400640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61f40b0ee0 0x7f61f415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f8a00640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61f40ce420 0x7f61f40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 -- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61f40ce420 msgr2=0x7f61f40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61f40ce420 0x7f61f40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 -- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61f40b0ee0 msgr2=0x7f61f415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61f40b0ee0 0x7f61f415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.152+0000 7f61f9e00640 1 -- 172.21.15.123:0/410997590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f40b1750 con 0x7f61f40cef40 2024-04-25T19:03:13.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61f9e00640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00ba10 tx=0x7f61ec00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec019070 con 0x7f61f40cef40 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ec00cb40 con 0x7f61f40cef40 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec014440 con 0x7f61f40cef40 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f40b1a30 con 0x7f61f40cef40 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.153+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f40b1ff0 con 0x7f61f40cef40 2024-04-25T19:03:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.154+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f4050740 con 0x7f61f40cef40 2024-04-25T19:03:13.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.156+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61ec01c070 con 0x7f61f40cef40 2024-04-25T19:03:13.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.157+0000 7f61f1600640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.157+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61ec095180 con 0x7f61f40cef40 2024-04-25T19:03:13.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.157+0000 7f61f9400640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.158+0000 7f61f9400640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 0x7f61d80769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61e8034000 tx=0x7f61e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.162+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61ec05f530 con 0x7f61f40cef40 2024-04-25T19:03:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.361+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61f40b14e0 con 0x7f61f40cef40 2024-04-25T19:03:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.362+0000 7f61f1600640 1 -- 172.21.15.123:0/410997590 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f61ec05ee30 con 0x7f61f40cef40 2024-04-25T19:03:13.364 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.364+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 msgr2=0x7f61d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.364+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 0x7f61d80769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61e8034000 tx=0x7f61e802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.364+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 msgr2=0x7f61f4153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.364+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00ba10 tx=0x7f61ec00bed0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 shutdown_connections 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61f40cef40 0x7f61f4153f40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61f40ce420 0x7f61f40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 --2- 172.21.15.123:0/410997590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61f40b0ee0 0x7f61f415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 >> 172.21.15.123:0/410997590 conn(0x7f61f409b4a0 msgr2=0x7f61f40cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 shutdown_connections 2024-04-25T19:03:13.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.365+0000 7f61fa800640 1 -- 172.21.15.123:0/410997590 wait complete. 2024-04-25T19:03:13.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:13.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.663+0000 7f9ace600640 1 Processor -- start 2024-04-25T19:03:13.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.663+0000 7f9ace600640 1 -- start start 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ace600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80cd140 0x7f9ac815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ace600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ac8162540 0x7f9ac80af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ace600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac80cc5f0 con 0x7f9ac806daa0 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ace600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac806deb0 con 0x7f9ac8162540 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ace600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac806d610 con 0x7f9ac80cd140 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ac7400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9ac806daa0 0x7f9ac80cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45364/0 (socket says 172.21.15.123:45364) 2024-04-25T19:03:13.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ac7400640 1 -- 172.21.15.123:0/511876228 learned_addr learned my addr 172.21.15.123:0/511876228 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.665+0000 7f9ac7e00640 1 --2- 172.21.15.123:0/511876228 >> v2:172.21.15.123:3300/0 conn(0x7f9ac8162540 0x7f9ac80af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9abfe00640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80cd140 0x7f9ac815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac6a00640 1 -- 172.21.15.123:0/511876228 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850610181 0 0) 0x7f9ac80cc5f0 con 0x7f9ac806daa0 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac6a00640 1 -- 172.21.15.123:0/511876228 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aac003520 con 0x7f9ac806daa0 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac7e00640 1 -- 172.21.15.123:0/511876228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80cd140 msgr2=0x7f9ac815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac7e00640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80cd140 0x7f9ac815dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac7e00640 1 -- 172.21.15.123:0/511876228 >> v1:172.21.15.123:6789/0 conn(0x7f9ac806daa0 legacy=0x7f9ac80cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.666+0000 7f9ac7e00640 1 -- 172.21.15.123:0/511876228 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac806ca00 con 0x7f9ac8162540 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.667+0000 7f9ac7e00640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac80af040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e862b3f42ea1a109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.667+0000 7f9ac6a00640 1 -- 172.21.15.123:0/511876228 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0007740 con 0x7f9ac8162540 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.667+0000 7f9ac6a00640 1 -- 172.21.15.123:0/511876228 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac00078e0 con 0x7f9ac8162540 2024-04-25T19:03:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.667+0000 7f9ac6a00640 1 -- 172.21.15.123:0/511876228 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0007cb0 con 0x7f9ac8162540 2024-04-25T19:03:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 -- 172.21.15.123:0/511876228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 msgr2=0x7f9ac80af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac80af040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac002a660 comp rx=0 tx=0).stop 2024-04-25T19:03:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 -- 172.21.15.123:0/511876228 shutdown_connections 2024-04-25T19:03:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac80af040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 --2- 172.21.15.123:0/511876228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80cd140 0x7f9ac815dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 -- 172.21.15.123:0/511876228 >> 172.21.15.123:0/511876228 conn(0x7f9ac80ce390 msgr2=0x7f9ac80ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 -- 172.21.15.123:0/511876228 shutdown_connections 2024-04-25T19:03:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.668+0000 7f9ace600640 1 -- 172.21.15.123:0/511876228 wait complete. 2024-04-25T19:03:13.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.671+0000 7f9ace600640 1 Processor -- start 2024-04-25T19:03:13.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.671+0000 7f9ace600640 1 -- start start 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac806daa0 0x7f9ac8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 0x7f9ac80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac806d2a0 con 0x7f9ac8162540 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac806cfc0 con 0x7f9ac8154270 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.672+0000 7f9ace600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac8162950 con 0x7f9ac806daa0 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34602/0 (socket says 172.21.15.123:34602) 2024-04-25T19:03:13.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 -- 172.21.15.123:0/1392184349 learned_addr learned my addr 172.21.15.123:0/1392184349 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9ac7400640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac806daa0 0x7f9ac8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9ac7e00640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 0x7f9ac80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 -- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac806daa0 msgr2=0x7f9ac8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac806daa0 0x7f9ac8151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 -- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 msgr2=0x7f9ac80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 0x7f9ac80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 -- 172.21.15.123:0/1392184349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac8156ae0 con 0x7f9ac8162540 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9ac7e00640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 0x7f9ac80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.673+0000 7f9abfe00640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d620 tx=0x7f9ab000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.674+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00125f0 con 0x7f9ac8162540 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.674+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0019070 con 0x7f9ac8162540 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.674+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00129d0 con 0x7f9ac8162540 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.674+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac8156dc0 con 0x7f9ac8162540 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.674+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac8157380 con 0x7f9ac8162540 2024-04-25T19:03:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.675+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac8076a40 con 0x7f9ac8162540 2024-04-25T19:03:13.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.676+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ab0012b70 con 0x7f9ac8162540 2024-04-25T19:03:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.677+0000 7f9abf400640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 0x7f9aa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.677+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9ab0095a10 con 0x7f9ac8162540 2024-04-25T19:03:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.677+0000 7f9ac7400640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 0x7f9aa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.678+0000 7f9ac7400640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 0x7f9aa40769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8000f60 tx=0x7f9ab8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.682+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ab005fdc0 con 0x7f9ac8162540 2024-04-25T19:03:13.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.881+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ac8069c60 con 0x7f9ac8162540 2024-04-25T19:03:13.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.882+0000 7f9abf400640 1 -- 172.21.15.123:0/1392184349 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f9ab005f6c0 con 0x7f9ac8162540 2024-04-25T19:03:13.883 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:13.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 msgr2=0x7f9aa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 0x7f9aa40769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8000f60 tx=0x7f9ab8009040 comp rx=0 tx=0).stop 2024-04-25T19:03:13.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 msgr2=0x7f9ac8158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d620 tx=0x7f9ab000dae0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 shutdown_connections 2024-04-25T19:03:13.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9aa40744b0 0x7f9aa40769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8154270 0x7f9ac80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac8162540 0x7f9ac8158ac0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 --2- 172.21.15.123:0/1392184349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac806daa0 0x7f9ac8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 >> 172.21.15.123:0/1392184349 conn(0x7f9ac80ce390 msgr2=0x7f9ac80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 shutdown_connections 2024-04-25T19:03:13.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:13.885+0000 7f9ace600640 1 -- 172.21.15.123:0/1392184349 wait complete. 2024-04-25T19:03:13.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:14.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.190+0000 7f13a9000640 1 Processor -- start 2024-04-25T19:03:14.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.190+0000 7f13a9000640 1 -- start start 2024-04-25T19:03:14.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.192+0000 7f13a9000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13a40cfc10 0x7f13a40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.192+0000 7f13a9000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.192+0000 7f13a9000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a4070a20 con 0x7f13a40b08c0 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.192+0000 7f13a9000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f13a406c780 con 0x7f13a40cfc10 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.192+0000 7f13a9000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13a406ca60 con 0x7f13a406fcd0 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13a40cfc10 0x7f13a40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13a40cfc10 0x7f13a40cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34610/0 (socket says 172.21.15.123:34610) 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 -- 172.21.15.123:0/3982995939 learned_addr learned my addr 172.21.15.123:0/3982995939 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a3e00640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 -- 172.21.15.123:0/3982995939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 msgr2=0x7f13a40700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a40700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 -- 172.21.15.123:0/3982995939 >> v1:172.21.15.146:6789/0 conn(0x7f13a40b08c0 legacy=0x7f13a40cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.193+0000 7f13a2a00640 1 -- 172.21.15.123:0/3982995939 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a406cd40 con 0x7f13a40cfc10 2024-04-25T19:03:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.194+0000 7f13a2a00640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40cfc10 0x7f13a40cd830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1390009870 tx=0x7f139002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba136f843bb1d5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.195+0000 7f13a2000640 1 -- 172.21.15.123:0/3982995939 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037070 con 0x7f13a40cfc10 2024-04-25T19:03:14.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.195+0000 7f13a2000640 1 -- 172.21.15.123:0/3982995939 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390032440 con 0x7f13a40cfc10 2024-04-25T19:03:14.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.195+0000 7f13a2000640 1 -- 172.21.15.123:0/3982995939 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390032810 con 0x7f13a40cfc10 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 -- 172.21.15.123:0/3982995939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40cfc10 msgr2=0x7f13a40cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40cfc10 0x7f13a40cd830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1390009870 tx=0x7f139002a270 comp rx=0 tx=0).stop 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 -- 172.21.15.123:0/3982995939 shutdown_connections 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a40700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 --2- 172.21.15.123:0/3982995939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40cfc10 0x7f13a40cd830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 -- 172.21.15.123:0/3982995939 >> 172.21.15.123:0/3982995939 conn(0x7f13a40d19f0 msgr2=0x7f13a40d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 -- 172.21.15.123:0/3982995939 shutdown_connections 2024-04-25T19:03:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.196+0000 7f13a9000640 1 -- 172.21.15.123:0/3982995939 wait complete. 2024-04-25T19:03:14.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.198+0000 7f13a9000640 1 Processor -- start 2024-04-25T19:03:14.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.198+0000 7f13a9000640 1 -- start start 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a4155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40b08c0 0x7f13a41442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406d3e0 con 0x7f13a40b4dd0 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406d260 con 0x7f13a40b08c0 2024-04-25T19:03:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a9000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13a406d560 con 0x7f13a406fcd0 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34620/0 (socket says 172.21.15.123:34620) 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a3e00640 1 -- 172.21.15.123:0/4109961420 learned_addr learned my addr 172.21.15.123:0/4109961420 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.200+0000 7f13a2a00640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40b08c0 0x7f13a41442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 -- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 msgr2=0x7f13a4155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a4155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 -- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40b08c0 msgr2=0x7f13a41442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40b08c0 0x7f13a41442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 -- 172.21.15.123:0/4109961420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a40b55e0 con 0x7f13a40b4dd0 2024-04-25T19:03:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a3e00640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e7f0 tx=0x7f139c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c017070 con 0x7f13a40b4dd0 2024-04-25T19:03:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c00c730 con 0x7f13a40b4dd0 2024-04-25T19:03:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a40b58c0 con 0x7f13a40b4dd0 2024-04-25T19:03:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00cb20 con 0x7f13a40b4dd0 2024-04-25T19:03:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.201+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a40b5e80 con 0x7f13a40b4dd0 2024-04-25T19:03:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.203+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4050740 con 0x7f13a40b4dd0 2024-04-25T19:03:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.204+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f139c01a070 con 0x7f13a40b4dd0 2024-04-25T19:03:14.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.205+0000 7f139b400640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 0x7f13840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.205+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f139c095640 con 0x7f13a40b4dd0 2024-04-25T19:03:14.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.205+0000 7f13a3400640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 0x7f13840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.206+0000 7f13a3400640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 0x7f13840769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13940012e0 tx=0x7f1394007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.209+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f139c05fbb0 con 0x7f13a40b4dd0 2024-04-25T19:03:14.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.408+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13a406fa40 con 0x7f13a40b4dd0 2024-04-25T19:03:14.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.409+0000 7f139b400640 1 -- 172.21.15.123:0/4109961420 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f139c05f4b0 con 0x7f13a40b4dd0 2024-04-25T19:03:14.411 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:14.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.411+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 msgr2=0x7f13840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.411+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 0x7f13840769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13940012e0 tx=0x7f1394007d50 comp rx=0 tx=0).stop 2024-04-25T19:03:14.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.411+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 msgr2=0x7f13a415cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.411+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e7f0 tx=0x7f139c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 shutdown_connections 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f13840744b0 0x7f13840769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a40b4dd0 0x7f13a415cda0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40b08c0 0x7f13a41442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 --2- 172.21.15.123:0/4109961420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a406fcd0 0x7f13a4155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 >> 172.21.15.123:0/4109961420 conn(0x7f13a40d19f0 msgr2=0x7f13a40cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 shutdown_connections 2024-04-25T19:03:14.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.412+0000 7f13a9000640 1 -- 172.21.15.123:0/4109961420 wait complete. 2024-04-25T19:03:14.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:14.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.707+0000 7f5c80e00640 1 Processor -- start 2024-04-25T19:03:14.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.707+0000 7f5c80e00640 1 -- start start 2024-04-25T19:03:14.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.709+0000 7f5c80e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c7c0d03a0 0x7f5c7c0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.709+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c1610f0 0x7f5c7c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.709+0000 7f5c80e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c0cf300 con 0x7f5c7c070820 2024-04-25T19:03:14.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.709+0000 7f5c80e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7c0cff10 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.709+0000 7f5c80e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cf5e0 con 0x7f5c7c1610f0 2024-04-25T19:03:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c7c0d03a0 0x7f5c7c0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c73e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5c7c070820 0x7f5c7c1609e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45366/0 (socket says 172.21.15.123:45366) 2024-04-25T19:03:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c73e00640 1 -- 172.21.15.123:0/278236704 learned_addr learned my addr 172.21.15.123:0/278236704 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c7a800640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c1610f0 0x7f5c7c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79e00640 1 -- 172.21.15.123:0/278236704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c1610f0 msgr2=0x7f5c7c06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79e00640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c1610f0 0x7f5c7c06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79e00640 1 -- 172.21.15.123:0/278236704 >> v1:172.21.15.123:6789/0 conn(0x7f5c7c070820 legacy=0x7f5c7c1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79400640 1 -- 172.21.15.123:0/278236704 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928085575 0 0) 0x7f5c7c0cf300 con 0x7f5c7c070820 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.710+0000 7f5c79e00640 1 -- 172.21.15.123:0/278236704 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c0cf8c0 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.711+0000 7f5c79e00640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0cbd80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0097a0 tx=0x7f5c6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd79171a3beda46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.712+0000 7f5c79400640 1 -- 172.21.15.123:0/278236704 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c037070 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.712+0000 7f5c79400640 1 -- 172.21.15.123:0/278236704 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c032440 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.712+0000 7f5c79400640 1 -- 172.21.15.123:0/278236704 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c032810 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.712+0000 7f5c80e00640 1 -- 172.21.15.123:0/278236704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 msgr2=0x7f5c7c0cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.712+0000 7f5c80e00640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0cbd80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0097a0 tx=0x7f5c6c02a180 comp rx=0 tx=0).stop 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 -- 172.21.15.123:0/278236704 shutdown_connections 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c1610f0 0x7f5c7c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 --2- 172.21.15.123:0/278236704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0cbd80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 -- 172.21.15.123:0/278236704 >> 172.21.15.123:0/278236704 conn(0x7f5c7c0ce390 msgr2=0x7f5c7c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 -- 172.21.15.123:0/278236704 shutdown_connections 2024-04-25T19:03:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.713+0000 7f5c80e00640 1 -- 172.21.15.123:0/278236704 wait complete. 2024-04-25T19:03:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.715+0000 7f5c80e00640 1 Processor -- start 2024-04-25T19:03:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.715+0000 7f5c80e00640 1 -- start start 2024-04-25T19:03:14.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c070820 0x7f5c7c14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c0ae7e0 0x7f5c7c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cdca0 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cc2c0 con 0x7f5c7c070820 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c80e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cde20 con 0x7f5c7c0ae7e0 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34636/0 (socket says 172.21.15.123:34636) 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c73e00640 1 -- 172.21.15.123:0/3900583803 learned_addr learned my addr 172.21.15.123:0/3900583803 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:14.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c79e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c070820 0x7f5c7c14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.717+0000 7f5c7a800640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c0ae7e0 0x7f5c7c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 -- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c0ae7e0 msgr2=0x7f5c7c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c0ae7e0 0x7f5c7c0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 -- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c070820 msgr2=0x7f5c7c14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c070820 0x7f5c7c14d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 -- 172.21.15.123:0/3900583803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c0b2910 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c73e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c460 tx=0x7f5c6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64018070 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c64007500 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004a20 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.718+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c0b1710 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.719+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c0b1cd0 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.720+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c15ce70 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.721+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c6401b070 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.722+0000 7f5c72000640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 0x7f5c58076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.722+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5c64094900 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.722+0000 7f5c79e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 0x7f5c58076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.723+0000 7f5c79e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 0x7f5c58076a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c034000 tx=0x7f5c6c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.726+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c7c15ce70 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.927+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5c7c076a40 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.928+0000 7f5c72000640 1 -- 172.21.15.123:0/3900583803 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5c6405e6e0 con 0x7f5c7c0d03a0 2024-04-25T19:03:14.930 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 msgr2=0x7f5c58076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 0x7f5c58076a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c034000 tx=0x7f5c6c02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 msgr2=0x7f5c7c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c460 tx=0x7f5c6400c920 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 shutdown_connections 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c58074500 0x7f5c58076a10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7c0ae7e0 0x7f5c7c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7c0d03a0 0x7f5c7c0caa80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 --2- 172.21.15.123:0/3900583803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7c070820 0x7f5c7c14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 >> 172.21.15.123:0/3900583803 conn(0x7f5c7c0ce390 msgr2=0x7f5c7c161150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 shutdown_connections 2024-04-25T19:03:14.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:14.931+0000 7f5c80e00640 1 -- 172.21.15.123:0/3900583803 wait complete. 2024-04-25T19:03:14.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:03:15.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.235+0000 7fa644a00640 1 Processor -- start 2024-04-25T19:03:15.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.235+0000 7fa644a00640 1 -- start start 2024-04-25T19:03:15.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.237+0000 7fa644a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa6400b5400 0x7fa64006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.237+0000 7fa644a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa640164220 0x7fa6400b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.237+0000 7fa644a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6400b59c0 con 0x7fa640163e10 2024-04-25T19:03:15.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.237+0000 7fa644a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa64015ab70 con 0x7fa640164220 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.237+0000 7fa644a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa6400b5ca0 con 0x7fa6400b5400 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa640164220 0x7fa6400b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa640164220 0x7fa6400b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49148/0 (socket says 172.21.15.123:49148) 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 -- 172.21.15.123:0/1355707927 learned_addr learned my addr 172.21.15.123:0/1355707927 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63de00640 1 --2- 172.21.15.123:0/1355707927 >> v2:172.21.15.146:3300/0 conn(0x7fa6400b5400 0x7fa64006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 -- 172.21.15.123:0/1355707927 >> v2:172.21.15.146:3300/0 conn(0x7fa6400b5400 msgr2=0x7fa64006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 --2- 172.21.15.123:0/1355707927 >> v2:172.21.15.146:3300/0 conn(0x7fa6400b5400 0x7fa64006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 -- 172.21.15.123:0/1355707927 >> v1:172.21.15.146:6789/0 conn(0x7fa640163e10 legacy=0x7fa6400b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.238+0000 7fa63e800640 1 -- 172.21.15.123:0/1355707927 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6400b5f80 con 0x7fa640164220 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.239+0000 7fa63e800640 1 --2- 172.21.15.123:0/1355707927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa640164220 0x7fa6400b0ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa640072a60 tx=0x7fa63802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94aba961d14e805f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.240+0000 7fa63d400640 1 -- 172.21.15.123:0/1355707927 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638036070 con 0x7fa640164220 2024-04-25T19:03:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.240+0000 7fa63d400640 1 -- 172.21.15.123:0/1355707927 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638031600 con 0x7fa640164220 2024-04-25T19:03:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.240+0000 7fa63d400640 1 -- 172.21.15.123:0/1355707927 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6380319d0 con 0x7fa640164220 2024-04-25T19:03:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.240+0000 7fa644a00640 1 -- 172.21.15.123:0/1355707927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa640164220 msgr2=0x7fa6400b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.240+0000 7fa644a00640 1 --2- 172.21.15.123:0/1355707927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa640164220 0x7fa6400b0ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa640072a60 tx=0x7fa63802a6f0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 -- 172.21.15.123:0/1355707927 shutdown_connections 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 --2- 172.21.15.123:0/1355707927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa640164220 0x7fa6400b0ad0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 --2- 172.21.15.123:0/1355707927 >> v2:172.21.15.146:3300/0 conn(0x7fa6400b5400 0x7fa64006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 -- 172.21.15.123:0/1355707927 >> 172.21.15.123:0/1355707927 conn(0x7fa6400d1ae0 msgr2=0x7fa6400d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 -- 172.21.15.123:0/1355707927 shutdown_connections 2024-04-25T19:03:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.241+0000 7fa644a00640 1 -- 172.21.15.123:0/1355707927 wait complete. 2024-04-25T19:03:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.244+0000 7fa644a00640 1 Processor -- start 2024-04-25T19:03:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.244+0000 7fa644a00640 1 -- start start 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa640163e10 0x7fa64006c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa640164220 0x7fa64006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa640070220 con 0x7fa6400b5400 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa6400b6540 con 0x7fa640164220 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa644a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa6400b1010 con 0x7fa640163e10 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49162/0 (socket says 172.21.15.123:49162) 2024-04-25T19:03:15.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 -- 172.21.15.123:0/1737239880 learned_addr learned my addr 172.21.15.123:0/1737239880 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa637e00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa640163e10 0x7fa64006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63e800640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa640164220 0x7fa64006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 -- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa640163e10 msgr2=0x7fa64006c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa640163e10 0x7fa64006c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 -- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa640164220 msgr2=0x7fa64006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa640164220 0x7fa64006cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.246+0000 7fa63de00640 1 -- 172.21.15.123:0/1737239880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64015be80 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa63de00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa63000d5d0 tx=0x7fa63000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630004240 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6300043e0 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630003830 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64015c160 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.247+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64015c720 con 0x7fa6400b5400 2024-04-25T19:03:15.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.248+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa640076a40 con 0x7fa6400b5400 2024-04-25T19:03:15.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.250+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa6300039d0 con 0x7fa6400b5400 2024-04-25T19:03:15.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.251+0000 7fa636000640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 0x7fa61c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.251+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa630095860 con 0x7fa6400b5400 2024-04-25T19:03:15.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.251+0000 7fa637e00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 0x7fa61c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.252+0000 7fa637e00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 0x7fa61c0769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa640161410 tx=0x7fa628008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.256+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa63005fca0 con 0x7fa6400b5400 2024-04-25T19:03:15.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.484+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa64015d510 con 0x7fa6400b5400 2024-04-25T19:03:15.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.485+0000 7fa636000640 1 -- 172.21.15.123:0/1737239880 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fa63005f5a0 con 0x7fa6400b5400 2024-04-25T19:03:15.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:15.487 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:03:15.487 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:03:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 msgr2=0x7fa61c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 0x7fa61c0769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa640161410 tx=0x7fa628008040 comp rx=0 tx=0).stop 2024-04-25T19:03:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 msgr2=0x7fa64015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa63000d5d0 tx=0x7fa63000da90 comp rx=0 tx=0).stop 2024-04-25T19:03:15.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 shutdown_connections 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa61c0744e0 0x7fa61c0769f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa640164220 0x7fa64006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa640163e10 0x7fa64006c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 --2- 172.21.15.123:0/1737239880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6400b5400 0x7fa64015d730 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.487+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 >> 172.21.15.123:0/1737239880 conn(0x7fa6400d1ae0 msgr2=0x7fa640068910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.488+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 shutdown_connections 2024-04-25T19:03:15.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.488+0000 7fa644a00640 1 -- 172.21.15.123:0/1737239880 wait complete. 2024-04-25T19:03:15.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:15.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.784+0000 7f93f5c00640 1 Processor -- start 2024-04-25T19:03:15.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.784+0000 7f93f5c00640 1 -- start start 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93f5c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93f015df00 0x7f93f006f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93f5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f93f00b5f00 0x7f93f00ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93f5c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93f00b4750 con 0x7f93f006f780 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93f5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f93f015e310 con 0x7f93f00b5f00 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93f5c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f93f00b5360 con 0x7f93f015df00 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93e7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f93f006f780 0x7f93f00b57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34462/0 (socket says 172.21.15.123:34462) 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93e7e00640 1 -- 172.21.15.123:0/1325902018 learned_addr learned my addr 172.21.15.123:0/1325902018 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93ef400640 1 --2- 172.21.15.123:0/1325902018 >> v2:172.21.15.123:3300/0 conn(0x7f93f00b5f00 0x7f93f00ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.787+0000 7f93eea00640 1 --2- 172.21.15.123:0/1325902018 >> v2:172.21.15.146:3300/0 conn(0x7f93f015df00 0x7f93f006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ee000640 1 -- 172.21.15.123:0/1325902018 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398525248 0 0) 0x7f93f00b4750 con 0x7f93f006f780 2024-04-25T19:03:15.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ee000640 1 -- 172.21.15.123:0/1325902018 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d4003520 con 0x7f93f006f780 2024-04-25T19:03:15.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ef400640 1 -- 172.21.15.123:0/1325902018 >> v2:172.21.15.146:3300/0 conn(0x7f93f015df00 msgr2=0x7f93f006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:15.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ef400640 1 --2- 172.21.15.123:0/1325902018 >> v2:172.21.15.146:3300/0 conn(0x7f93f015df00 0x7f93f006f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ef400640 1 -- 172.21.15.123:0/1325902018 >> v1:172.21.15.123:6789/0 conn(0x7f93f006f780 legacy=0x7f93f00b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.788+0000 7f93ef400640 1 -- 172.21.15.123:0/1325902018 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f00b4a30 con 0x7f93f00b5f00 2024-04-25T19:03:15.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.789+0000 7f93ef400640 1 --2- 172.21.15.123:0/1325902018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00ae7e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93f006c680 tx=0x7f93e802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da54eeea92e84be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.789+0000 7f93ee000640 1 -- 172.21.15.123:0/1325902018 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8007740 con 0x7f93f00b5f00 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.789+0000 7f93ee000640 1 -- 172.21.15.123:0/1325902018 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e80078e0 con 0x7f93f00b5f00 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.789+0000 7f93ee000640 1 -- 172.21.15.123:0/1325902018 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8007c90 con 0x7f93f00b5f00 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 -- 172.21.15.123:0/1325902018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 msgr2=0x7f93f00ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 --2- 172.21.15.123:0/1325902018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00ae7e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93f006c680 tx=0x7f93e802a5f0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 -- 172.21.15.123:0/1325902018 shutdown_connections 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 --2- 172.21.15.123:0/1325902018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00ae7e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 --2- 172.21.15.123:0/1325902018 >> v2:172.21.15.146:3300/0 conn(0x7f93f015df00 0x7f93f006f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 -- 172.21.15.123:0/1325902018 >> 172.21.15.123:0/1325902018 conn(0x7f93f00d1ae0 msgr2=0x7f93f00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 -- 172.21.15.123:0/1325902018 shutdown_connections 2024-04-25T19:03:15.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.790+0000 7f93f5c00640 1 -- 172.21.15.123:0/1325902018 wait complete. 2024-04-25T19:03:15.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.792+0000 7f93f5c00640 1 Processor -- start 2024-04-25T19:03:15.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.793+0000 7f93f5c00640 1 -- start start 2024-04-25T19:03:15.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93f006f780 0x7f93f0156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93f015df00 0x7f93f00d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00aed20 con 0x7f93f00b5f00 2024-04-25T19:03:15.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00b4ff0 con 0x7f93f006f780 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93f5c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93f00aeea0 con 0x7f93f015df00 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49176/0 (socket says 172.21.15.123:49176) 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.794+0000 7f93e7e00640 1 -- 172.21.15.123:0/2573278562 learned_addr learned my addr 172.21.15.123:0/2573278562 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93eea00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93f006f780 0x7f93f0156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93ef400640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93f015df00 0x7f93f00d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 -- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93f015df00 msgr2=0x7f93f00d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93f015df00 0x7f93f00d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 -- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93f006f780 msgr2=0x7f93f0156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93f006f780 0x7f93f0156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 -- 172.21.15.123:0/2573278562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f00cf1f0 con 0x7f93f00b5f00 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e7e00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c400 tx=0x7f93d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017070 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.795+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d8005d00 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.796+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f00cf4d0 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.796+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8007310 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.796+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f00cfa90 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.797+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f0068f10 con 0x7f93f00b5f00 2024-04-25T19:03:15.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.798+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f93d801a070 con 0x7f93f00b5f00 2024-04-25T19:03:15.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.799+0000 7f93e6a00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 0x7f93cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:15.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.799+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f93d8094a20 con 0x7f93f00b5f00 2024-04-25T19:03:15.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.799+0000 7f93eea00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 0x7f93cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:15.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.800+0000 7f93eea00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 0x7f93cc0769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009a30 tx=0x7f93e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:15.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:15.803+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93f0068f10 con 0x7f93f00b5f00 2024-04-25T19:03:16.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.007+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93f0076a40 con 0x7f93f00b5f00 2024-04-25T19:03:16.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.009+0000 7f93e6a00640 1 -- 172.21.15.123:0/2573278562 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f93d805edd0 con 0x7f93f00b5f00 2024-04-25T19:03:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:16.013 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 msgr2=0x7f93cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 0x7f93cc0769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009a30 tx=0x7f93e0008040 comp rx=0 tx=0).stop 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 msgr2=0x7f93f00d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c400 tx=0x7f93d800c8c0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 shutdown_connections 2024-04-25T19:03:16.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f93cc0744b0 0x7f93cc0769c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93f015df00 0x7f93f00d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f00b5f00 0x7f93f00d0290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 --2- 172.21.15.123:0/2573278562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93f006f780 0x7f93f0156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.012+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 >> 172.21.15.123:0/2573278562 conn(0x7f93f00d1ae0 msgr2=0x7f93f00684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.013+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 shutdown_connections 2024-04-25T19:03:16.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.013+0000 7f93f5c00640 1 -- 172.21.15.123:0/2573278562 wait complete. 2024-04-25T19:03:16.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:16.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:16.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.321+0000 7f64e3e00640 1 Processor -- start 2024-04-25T19:03:16.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.321+0000 7f64e3e00640 1 -- start start 2024-04-25T19:03:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e40cfff0 0x7f64e415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e3e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f64e4143040 0x7f64e406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e3e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e415e2e0 con 0x7f64e406dac0 2024-04-25T19:03:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f64e40d1200 con 0x7f64e40cfff0 2024-04-25T19:03:16.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e3e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f64e406ded0 con 0x7f64e4143040 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.322+0000 7f64e2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e40cfff0 0x7f64e415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e40cfff0 0x7f64e415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49184/0 (socket says 172.21.15.123:49184) 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 -- 172.21.15.123:0/1807716328 learned_addr learned my addr 172.21.15.123:0/1807716328 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64dbe00640 1 --2- 172.21.15.123:0/1807716328 >> v2:172.21.15.146:3300/0 conn(0x7f64e4143040 0x7f64e406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 -- 172.21.15.123:0/1807716328 >> v2:172.21.15.146:3300/0 conn(0x7f64e4143040 msgr2=0x7f64e406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:16.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 --2- 172.21.15.123:0/1807716328 >> v2:172.21.15.146:3300/0 conn(0x7f64e4143040 0x7f64e406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 -- 172.21.15.123:0/1807716328 >> v1:172.21.15.146:6789/0 conn(0x7f64e406dac0 legacy=0x7f64e4160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:16.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.323+0000 7f64e2a00640 1 -- 172.21.15.123:0/1807716328 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e40d05f0 con 0x7f64e40cfff0 2024-04-25T19:03:16.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.324+0000 7f64e2a00640 1 --2- 172.21.15.123:0/1807716328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e40cfff0 0x7f64e415dda0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64e4157970 tx=0x7f64d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2033eadce0315d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.324+0000 7f64e2000640 1 -- 172.21.15.123:0/1807716328 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4037070 con 0x7f64e40cfff0 2024-04-25T19:03:16.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.325+0000 7f64e2000640 1 -- 172.21.15.123:0/1807716328 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d4032440 con 0x7f64e40cfff0 2024-04-25T19:03:16.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.325+0000 7f64e2000640 1 -- 172.21.15.123:0/1807716328 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4032810 con 0x7f64e40cfff0 2024-04-25T19:03:16.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.325+0000 7f64e3e00640 1 -- 172.21.15.123:0/1807716328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e40cfff0 msgr2=0x7f64e415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.325+0000 7f64e3e00640 1 --2- 172.21.15.123:0/1807716328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e40cfff0 0x7f64e415dda0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64e4157970 tx=0x7f64d402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 -- 172.21.15.123:0/1807716328 shutdown_connections 2024-04-25T19:03:16.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 --2- 172.21.15.123:0/1807716328 >> v2:172.21.15.146:3300/0 conn(0x7f64e4143040 0x7f64e406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 --2- 172.21.15.123:0/1807716328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e40cfff0 0x7f64e415dda0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 -- 172.21.15.123:0/1807716328 >> 172.21.15.123:0/1807716328 conn(0x7f64e4073090 msgr2=0x7f64e40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:16.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 -- 172.21.15.123:0/1807716328 shutdown_connections 2024-04-25T19:03:16.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.326+0000 7f64e3e00640 1 -- 172.21.15.123:0/1807716328 wait complete. 2024-04-25T19:03:16.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.328+0000 7f64e3e00640 1 Processor -- start 2024-04-25T19:03:16.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.328+0000 7f64e3e00640 1 -- start start 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64e406dac0 0x7f64e40cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64e40cfff0 0x7f64e4140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64e40d0e90 con 0x7f64e4143040 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f64e40d0bb0 con 0x7f64e406dac0 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f64e4143450 con 0x7f64e40cfff0 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49196/0 (socket says 172.21.15.123:49196) 2024-04-25T19:03:16.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 -- 172.21.15.123:0/309245852 learned_addr learned my addr 172.21.15.123:0/309245852 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e2a00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64e406dac0 0x7f64e40cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64dbe00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64e40cfff0 0x7f64e4140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 -- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64e40cfff0 msgr2=0x7f64e4140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64e40cfff0 0x7f64e4140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 -- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64e406dac0 msgr2=0x7f64e40cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64e406dac0 0x7f64e40cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 -- 172.21.15.123:0/309245852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e40cd120 con 0x7f64e4143040 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3400640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b970 tx=0x7f64dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc019070 con 0x7f64e4143040 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64dc00cb40 con 0x7f64e4143040 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc014440 con 0x7f64e4143040 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e40b4ab0 con 0x7f64e4143040 2024-04-25T19:03:16.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.331+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e40b5070 con 0x7f64e4143040 2024-04-25T19:03:16.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.332+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e4050740 con 0x7f64e4143040 2024-04-25T19:03:16.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.334+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f64dc01c070 con 0x7f64e4143040 2024-04-25T19:03:16.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.334+0000 7f64daa00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 0x7f64c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.335+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f64dc095220 con 0x7f64e4143040 2024-04-25T19:03:16.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.335+0000 7f64e2a00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 0x7f64c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.335+0000 7f64e2a00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 0x7f64c00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64d4034000 tx=0x7f64d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.339+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64dc05f660 con 0x7f64e4143040 2024-04-25T19:03:16.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.533+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64e40cceb0 con 0x7f64e4143040 2024-04-25T19:03:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.535+0000 7f64daa00640 1 -- 172.21.15.123:0/309245852 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f64dc05ef60 con 0x7f64e4143040 2024-04-25T19:03:16.537 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 msgr2=0x7f64c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 0x7f64c00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64d4034000 tx=0x7f64d402e000 comp rx=0 tx=0).stop 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 msgr2=0x7f64e40aa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b970 tx=0x7f64dc00be30 comp rx=0 tx=0).stop 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 shutdown_connections 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64c00744b0 0x7f64c00769c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4143040 0x7f64e40aa6f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64e40cfff0 0x7f64e4140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 --2- 172.21.15.123:0/309245852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64e406dac0 0x7f64e40cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 >> 172.21.15.123:0/309245852 conn(0x7f64e4073090 msgr2=0x7f64e40cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:16.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 shutdown_connections 2024-04-25T19:03:16.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.537+0000 7f64e3e00640 1 -- 172.21.15.123:0/309245852 wait complete. 2024-04-25T19:03:16.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:16.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.830+0000 7fb9f1600640 1 Processor -- start 2024-04-25T19:03:16.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.830+0000 7fb9f1600640 1 -- start start 2024-04-25T19:03:16.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.832+0000 7fb9f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9ec070820 0x7fb9ec1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.832+0000 7fb9f1600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb9ec0cdca0 0x7fb9ec160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.832+0000 7fb9f1600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9ec0cf300 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.832+0000 7fb9f1600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ec0cff10 con 0x7fb9ec070820 2024-04-25T19:03:16.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.832+0000 7fb9f1600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ec0cf5e0 con 0x7fb9ec0cdca0 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9ec070820 0x7fb9ec1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9ec070820 0x7fb9ec1609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49212/0 (socket says 172.21.15.123:49212) 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 -- 172.21.15.123:0/2810608736 learned_addr learned my addr 172.21.15.123:0/2810608736 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9eb400640 1 --2- 172.21.15.123:0/2810608736 >> v2:172.21.15.146:3300/0 conn(0x7fb9ec0cdca0 0x7fb9ec160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 -- 172.21.15.123:0/2810608736 >> v2:172.21.15.146:3300/0 conn(0x7fb9ec0cdca0 msgr2=0x7fb9ec160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:16.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 --2- 172.21.15.123:0/2810608736 >> v2:172.21.15.146:3300/0 conn(0x7fb9ec0cdca0 0x7fb9ec160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 -- 172.21.15.123:0/2810608736 >> v1:172.21.15.146:6789/0 conn(0x7fb9ec0d03a0 legacy=0x7fb9ec0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.833+0000 7fb9ea000640 1 -- 172.21.15.123:0/2810608736 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec0cf8c0 con 0x7fb9ec070820 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.834+0000 7fb9ea000640 1 --2- 172.21.15.123:0/2810608736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec070820 0x7fb9ec1609e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0d0ac0 tx=0x7fb9d802c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ac675f9ebf28b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.835+0000 7fb9e9600640 1 -- 172.21.15.123:0/2810608736 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8038070 con 0x7fb9ec070820 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.835+0000 7fb9e9600640 1 -- 172.21.15.123:0/2810608736 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d8031710 con 0x7fb9ec070820 2024-04-25T19:03:16.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.835+0000 7fb9e9600640 1 -- 172.21.15.123:0/2810608736 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8031ae0 con 0x7fb9ec070820 2024-04-25T19:03:16.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 -- 172.21.15.123:0/2810608736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec070820 msgr2=0x7fb9ec1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 --2- 172.21.15.123:0/2810608736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec070820 0x7fb9ec1609e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0d0ac0 tx=0x7fb9d802c440 comp rx=0 tx=0).stop 2024-04-25T19:03:16.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 -- 172.21.15.123:0/2810608736 shutdown_connections 2024-04-25T19:03:16.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 --2- 172.21.15.123:0/2810608736 >> v2:172.21.15.146:3300/0 conn(0x7fb9ec0cdca0 0x7fb9ec160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 --2- 172.21.15.123:0/2810608736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec070820 0x7fb9ec1609e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 -- 172.21.15.123:0/2810608736 >> 172.21.15.123:0/2810608736 conn(0x7fb9ec0ce390 msgr2=0x7fb9ec0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:16.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 -- 172.21.15.123:0/2810608736 shutdown_connections 2024-04-25T19:03:16.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.836+0000 7fb9f1600640 1 -- 172.21.15.123:0/2810608736 wait complete. 2024-04-25T19:03:16.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.838+0000 7fb9f1600640 1 Processor -- start 2024-04-25T19:03:16.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.838+0000 7fb9f1600640 1 -- start start 2024-04-25T19:03:16.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 0x7fb9ec0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9ec0cdca0 0x7fb9ec0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec06f190 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec06f010 con 0x7fb9ec070820 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9f1600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ec06f310 con 0x7fb9ec0cdca0 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49222/0 (socket says 172.21.15.123:49222) 2024-04-25T19:03:16.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 -- 172.21.15.123:0/515863864 learned_addr learned my addr 172.21.15.123:0/515863864 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eaa00640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 0x7fb9ec0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9ea000640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9ec0cdca0 0x7fb9ec0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 -- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9ec0cdca0 msgr2=0x7fb9ec0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9ec0cdca0 0x7fb9ec0cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 -- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 msgr2=0x7fb9ec0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 0x7fb9ec0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:16.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eb400640 1 -- 172.21.15.123:0/515863864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec158f30 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.840+0000 7fb9eaa00640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 0x7fb9ec0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9eb400640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009fd0 tx=0x7fb9e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4014420 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4019070 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4014800 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ec159210 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.841+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ec1573e0 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.842+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec076a40 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.843+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb9e40149a0 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.846+0000 7fb9e2a00640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 0x7fb9cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:16.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.846+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb9e4095dc0 con 0x7fb9ec0d03a0 2024-04-25T19:03:16.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.846+0000 7fb9eaa00640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 0x7fb9cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:16.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.846+0000 7fb9eaa00640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 0x7fb9cc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b0fa0 tx=0x7fb9dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:16.850+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9e4060210 con 0x7fb9ec0d03a0 2024-04-25T19:03:17.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.052+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9ec069c60 con 0x7fb9ec0d03a0 2024-04-25T19:03:17.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.053+0000 7fb9e2a00640 1 -- 172.21.15.123:0/515863864 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb9e405fb10 con 0x7fb9ec0d03a0 2024-04-25T19:03:17.055 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.055+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 msgr2=0x7fb9cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.055+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 0x7fb9cc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b0fa0 tx=0x7fb9dc008040 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.055+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 msgr2=0x7fb9ec157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.055+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009fd0 tx=0x7fb9e400e4b0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 shutdown_connections 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9cc0744b0 0x7fb9cc0769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9ec0d03a0 0x7fb9ec157be0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9ec0cdca0 0x7fb9ec0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 --2- 172.21.15.123:0/515863864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9ec070820 0x7fb9ec0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 >> 172.21.15.123:0/515863864 conn(0x7fb9ec0ce390 msgr2=0x7fb9ec0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:17.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 shutdown_connections 2024-04-25T19:03:17.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.056+0000 7fb9f1600640 1 -- 172.21.15.123:0/515863864 wait complete. 2024-04-25T19:03:17.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.335+0000 7f2aae000640 1 Processor -- start 2024-04-25T19:03:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.335+0000 7f2aae000640 1 -- start start 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aae000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80b24a0 0x7f2aa80d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aae000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2aa80d14e0 0x7f2aa80d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aae000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa80d19a0 con 0x7f2aa80ce150 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aae000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa80b19c0 con 0x7f2aa80d14e0 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aae000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa80b1ca0 con 0x7f2aa80b24a0 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aa7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2aa80d14e0 0x7f2aa80d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aad600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2aa80ce150 0x7f2aa80ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34464/0 (socket says 172.21.15.123:34464) 2024-04-25T19:03:17.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.338+0000 7f2aad600640 1 -- 172.21.15.123:0/2734335428 learned_addr learned my addr 172.21.15.123:0/2734335428 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:17.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.339+0000 7f2aacc00640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80b24a0 0x7f2aa80d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.339+0000 7f2aa7e00640 1 -- 172.21.15.123:0/2734335428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80b24a0 msgr2=0x7f2aa80d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:17.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.339+0000 7f2aa7e00640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80b24a0 0x7f2aa80d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.339+0000 7f2aa7e00640 1 -- 172.21.15.123:0/2734335428 >> v1:172.21.15.123:6789/0 conn(0x7f2aa80ce150 legacy=0x7f2aa80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.339+0000 7f2aa7e00640 1 -- 172.21.15.123:0/2734335428 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa80b1f80 con 0x7f2aa80d14e0 2024-04-25T19:03:17.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.340+0000 7f2aa7e00640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa80d14e0 0x7f2aa80d1dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a72ae6fb6813a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.340+0000 7f2aa7400640 1 -- 172.21.15.123:0/2734335428 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94037070 con 0x7f2aa80d14e0 2024-04-25T19:03:17.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.340+0000 7f2aa7400640 1 -- 172.21.15.123:0/2734335428 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a94032440 con 0x7f2aa80d14e0 2024-04-25T19:03:17.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.340+0000 7f2aa7400640 1 -- 172.21.15.123:0/2734335428 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94032810 con 0x7f2aa80d14e0 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 -- 172.21.15.123:0/2734335428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa80d14e0 msgr2=0x7f2aa80d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa80d14e0 0x7f2aa80d1dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402a180 comp rx=0 tx=0).stop 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 -- 172.21.15.123:0/2734335428 shutdown_connections 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa80d14e0 0x7f2aa80d1dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 --2- 172.21.15.123:0/2734335428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80b24a0 0x7f2aa80d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 -- 172.21.15.123:0/2734335428 >> 172.21.15.123:0/2734335428 conn(0x7f2aa809b4a0 msgr2=0x7f2aa80b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 -- 172.21.15.123:0/2734335428 shutdown_connections 2024-04-25T19:03:17.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.341+0000 7f2aae000640 1 -- 172.21.15.123:0/2734335428 wait complete. 2024-04-25T19:03:17.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.344+0000 7f2aae000640 1 Processor -- start 2024-04-25T19:03:17.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.344+0000 7f2aae000640 1 -- start start 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.345+0000 7f2aae000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2aa80b24a0 0x7f2aa814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.345+0000 7f2aae000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 0x7f2aa815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aae000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa80d2490 con 0x7f2aa815e700 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aae000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa80d2310 con 0x7f2aa80b24a0 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aae000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa80d2610 con 0x7f2aa80ce150 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49234/0 (socket says 172.21.15.123:49234) 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 -- 172.21.15.123:0/228198966 learned_addr learned my addr 172.21.15.123:0/228198966 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aacc00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2aa80b24a0 0x7f2aa814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aa7e00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 0x7f2aa815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 -- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 msgr2=0x7f2aa815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 0x7f2aa815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 -- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2aa80b24a0 msgr2=0x7f2aa814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2aa80b24a0 0x7f2aa814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aad600640 1 -- 172.21.15.123:0/228198966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa8152490 con 0x7f2aa815e700 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.346+0000 7f2aa7e00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 0x7f2aa815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aad600640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000ca70 tx=0x7f2aa000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0017070 con 0x7f2aa815e700 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa000abd0 con 0x7f2aa815e700 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0012400 con 0x7f2aa815e700 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa814f1e0 con 0x7f2aa815e700 2024-04-25T19:03:17.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.347+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa814c5e0 con 0x7f2aa815e700 2024-04-25T19:03:17.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.348+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa814ef50 con 0x7f2aa815e700 2024-04-25T19:03:17.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.349+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2aa001a070 con 0x7f2aa815e700 2024-04-25T19:03:17.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.350+0000 7f2aa4c00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 0x7f2a880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.350+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2aa0095320 con 0x7f2aa815e700 2024-04-25T19:03:17.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.351+0000 7f2aacc00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 0x7f2a880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.351+0000 7f2aacc00640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 0x7f2a880769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.355+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2aa005f6d0 con 0x7f2aa815e700 2024-04-25T19:03:17.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.539+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2aa80b37b0 con 0x7f2aa815e700 2024-04-25T19:03:17.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.540+0000 7f2aa4c00640 1 -- 172.21.15.123:0/228198966 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f2aa005efd0 con 0x7f2aa815e700 2024-04-25T19:03:17.542 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.542+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 msgr2=0x7f2a880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.542+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 0x7f2a880769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98008040 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.542+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 msgr2=0x7f2aa814b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.542+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000ca70 tx=0x7f2aa000cf30 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 shutdown_connections 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2a880744b0 0x7f2a880769c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa815e700 0x7f2aa814b990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2aa80ce150 0x7f2aa815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 --2- 172.21.15.123:0/228198966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2aa80b24a0 0x7f2aa814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 >> 172.21.15.123:0/228198966 conn(0x7f2aa809b4a0 msgr2=0x7f2aa80afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 shutdown_connections 2024-04-25T19:03:17.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.543+0000 7f2aae000640 1 -- 172.21.15.123:0/228198966 wait complete. 2024-04-25T19:03:17.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:17.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.839+0000 7ff426400640 1 Processor -- start 2024-04-25T19:03:17.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.839+0000 7ff426400640 1 -- start start 2024-04-25T19:03:17.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.841+0000 7ff426400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4200cd140 0x7ff42015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.841+0000 7ff426400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4200cc5f0 con 0x7ff420162540 2024-04-25T19:03:17.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.841+0000 7ff426400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42006ca00 con 0x7ff42006daa0 2024-04-25T19:03:17.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.841+0000 7ff426400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff42006deb0 con 0x7ff4200cd140 2024-04-25T19:03:17.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff420162540 0x7ff4200af040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34468/0 (socket says 172.21.15.123:34468) 2024-04-25T19:03:17.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41fe00640 1 -- 172.21.15.123:0/3303241134 learned_addr learned my addr 172.21.15.123:0/3303241134 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:17.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41ea00640 1 --2- 172.21.15.123:0/3303241134 >> v2:172.21.15.123:3300/0 conn(0x7ff4200cd140 0x7ff42015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41e000640 1 -- 172.21.15.123:0/3303241134 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994932845 0 0) 0x7ff4200cc5f0 con 0x7ff420162540 2024-04-25T19:03:17.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41e000640 1 -- 172.21.15.123:0/3303241134 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff404003520 con 0x7ff420162540 2024-04-25T19:03:17.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41ea00640 1 -- 172.21.15.123:0/3303241134 >> v1:172.21.15.146:6789/0 conn(0x7ff42006daa0 legacy=0x7ff4200cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41ea00640 1 -- 172.21.15.123:0/3303241134 >> v1:172.21.15.123:6789/0 conn(0x7ff420162540 legacy=0x7ff4200af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.842+0000 7ff41ea00640 1 -- 172.21.15.123:0/3303241134 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42006cce0 con 0x7ff4200cd140 2024-04-25T19:03:17.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.843+0000 7ff41ea00640 1 --2- 172.21.15.123:0/3303241134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4200cd140 0x7ff42015dfc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0097a0 tx=0x7ff40c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddad05e49e0b5712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.844+0000 7ff41e000640 1 -- 172.21.15.123:0/3303241134 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c037070 con 0x7ff4200cd140 2024-04-25T19:03:17.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.844+0000 7ff41e000640 1 -- 172.21.15.123:0/3303241134 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c032440 con 0x7ff4200cd140 2024-04-25T19:03:17.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.844+0000 7ff41e000640 1 -- 172.21.15.123:0/3303241134 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c032810 con 0x7ff4200cd140 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.844+0000 7ff426400640 1 -- 172.21.15.123:0/3303241134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4200cd140 msgr2=0x7ff42015dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.844+0000 7ff426400640 1 --2- 172.21.15.123:0/3303241134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4200cd140 0x7ff42015dfc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0097a0 tx=0x7ff40c02a230 comp rx=0 tx=0).stop 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.845+0000 7ff426400640 1 -- 172.21.15.123:0/3303241134 shutdown_connections 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.845+0000 7ff426400640 1 --2- 172.21.15.123:0/3303241134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4200cd140 0x7ff42015dfc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.845+0000 7ff426400640 1 -- 172.21.15.123:0/3303241134 >> 172.21.15.123:0/3303241134 conn(0x7ff4200ce390 msgr2=0x7ff4200ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.845+0000 7ff426400640 1 -- 172.21.15.123:0/3303241134 shutdown_connections 2024-04-25T19:03:17.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.845+0000 7ff426400640 1 -- 172.21.15.123:0/3303241134 wait complete. 2024-04-25T19:03:17.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.847+0000 7ff426400640 1 Processor -- start 2024-04-25T19:03:17.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.847+0000 7ff426400640 1 -- start start 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420162540 0x7ff42015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff42015fb20 0x7ff42015ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff42006d2a0 con 0x7ff42006daa0 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff42006d610 con 0x7ff420162540 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff426400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff420162950 con 0x7ff42015fb20 2024-04-25T19:03:17.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49252/0 (socket says 172.21.15.123:49252) 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 -- 172.21.15.123:0/594193836 learned_addr learned my addr 172.21.15.123:0/594193836 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41ea00640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420162540 0x7ff42015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 -- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff42015fb20 msgr2=0x7ff42015ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff42015fb20 0x7ff42015ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 -- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420162540 msgr2=0x7ff42015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420162540 0x7ff42015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.849+0000 7ff41f400640 1 -- 172.21.15.123:0/594193836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff420155600 con 0x7ff42006daa0 2024-04-25T19:03:17.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff41f400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c470 tx=0x7ff41000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4100052d0 con 0x7ff42006daa0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410005490 con 0x7ff42006daa0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff420155a70 con 0x7ff42006daa0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410007260 con 0x7ff42006daa0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.850+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4200b54e0 con 0x7ff42006daa0 2024-04-25T19:03:17.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.851+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff420076a40 con 0x7ff42006daa0 2024-04-25T19:03:17.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.853+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff410007400 con 0x7ff42006daa0 2024-04-25T19:03:17.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.853+0000 7ff417400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 0x7ff4000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:17.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.854+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff4100949a0 con 0x7ff42006daa0 2024-04-25T19:03:17.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.854+0000 7ff41ea00640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 0x7ff4000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:17.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.855+0000 7ff41ea00640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 0x7ff4000769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff40c034000 tx=0x7ff40c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:17.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:17.859+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff41005ede0 con 0x7ff42006daa0 2024-04-25T19:03:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.058+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff420069c60 con 0x7ff42006daa0 2024-04-25T19:03:18.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.058+0000 7ff417400640 1 -- 172.21.15.123:0/594193836 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7ff41005e6e0 con 0x7ff42006daa0 2024-04-25T19:03:18.060 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 msgr2=0x7ff4000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 0x7ff4000769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff40c034000 tx=0x7ff40c02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 msgr2=0x7ff4201492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c470 tx=0x7ff41000c930 comp rx=0 tx=0).stop 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 shutdown_connections 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff4000744b0 0x7ff4000769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff42015fb20 0x7ff42015ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420162540 0x7ff42015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 --2- 172.21.15.123:0/594193836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff42006daa0 0x7ff4201492c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 >> 172.21.15.123:0/594193836 conn(0x7ff4200ce390 msgr2=0x7ff42015d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 shutdown_connections 2024-04-25T19:03:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.061+0000 7ff426400640 1 -- 172.21.15.123:0/594193836 wait complete. 2024-04-25T19:03:18.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:03:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.342+0000 7f43d1400640 1 Processor -- start 2024-04-25T19:03:18.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.342+0000 7f43d1400640 1 -- start start 2024-04-25T19:03:18.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.344+0000 7f43d1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43cc06dc90 0x7f43cc0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.344+0000 7f43d1400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc1456e0 con 0x7f43cc1450d0 2024-04-25T19:03:18.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.344+0000 7f43d1400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc1459c0 con 0x7f43cc0b0af0 2024-04-25T19:03:18.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.344+0000 7f43d1400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc1462f0 con 0x7f43cc06dc90 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43cb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43cc06dc90 0x7f43cc0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43cbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f43cc1450d0 0x7f43cc06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34472/0 (socket says 172.21.15.123:34472) 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43cbe00640 1 -- 172.21.15.123:0/4181842429 learned_addr learned my addr 172.21.15.123:0/4181842429 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43cb400640 1 -- 172.21.15.123:0/4181842429 >> v1:172.21.15.146:6789/0 conn(0x7f43cc0b0af0 legacy=0x7f43cc146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43caa00640 1 -- 172.21.15.123:0/4181842429 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237511118 0 0) 0x7f43cc1456e0 con 0x7f43cc1450d0 2024-04-25T19:03:18.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.345+0000 7f43cb400640 1 -- 172.21.15.123:0/4181842429 >> v1:172.21.15.123:6789/0 conn(0x7f43cc1450d0 legacy=0x7f43cc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.346+0000 7f43cb400640 1 -- 172.21.15.123:0/4181842429 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc145ca0 con 0x7f43cc06dc90 2024-04-25T19:03:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.346+0000 7f43cb400640 1 --2- 172.21.15.123:0/4181842429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc06dc90 0x7f43cc0b05b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f43cc146470 tx=0x7f43b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeb85ae558054ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43caa00640 1 -- 172.21.15.123:0/4181842429 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8037070 con 0x7f43cc06dc90 2024-04-25T19:03:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43caa00640 1 -- 172.21.15.123:0/4181842429 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8032440 con 0x7f43cc06dc90 2024-04-25T19:03:18.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43caa00640 1 -- 172.21.15.123:0/4181842429 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8032810 con 0x7f43cc06dc90 2024-04-25T19:03:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43d1400640 1 -- 172.21.15.123:0/4181842429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc06dc90 msgr2=0x7f43cc0b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43d1400640 1 --2- 172.21.15.123:0/4181842429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc06dc90 0x7f43cc0b05b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f43cc146470 tx=0x7f43b802a210 comp rx=0 tx=0).stop 2024-04-25T19:03:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43d1400640 1 -- 172.21.15.123:0/4181842429 shutdown_connections 2024-04-25T19:03:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43d1400640 1 --2- 172.21.15.123:0/4181842429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc06dc90 0x7f43cc0b05b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.347+0000 7f43d1400640 1 -- 172.21.15.123:0/4181842429 >> 172.21.15.123:0/4181842429 conn(0x7f43cc073200 msgr2=0x7f43cc073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.348+0000 7f43d1400640 1 -- 172.21.15.123:0/4181842429 shutdown_connections 2024-04-25T19:03:18.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.348+0000 7f43d1400640 1 -- 172.21.15.123:0/4181842429 wait complete. 2024-04-25T19:03:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.350+0000 7f43d1400640 1 Processor -- start 2024-04-25T19:03:18.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.350+0000 7f43d1400640 1 -- start start 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 0x7f43cc0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43cc0b2bb0 0x7f43cc162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc0b1080 con 0x7f43cc1450d0 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc0b0f00 con 0x7f43cc0b0af0 2024-04-25T19:03:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.351+0000 7f43d1400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc1471e0 con 0x7f43cc0b2bb0 2024-04-25T19:03:18.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49282/0 (socket says 172.21.15.123:49282) 2024-04-25T19:03:18.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 -- 172.21.15.123:0/1335191131 learned_addr learned my addr 172.21.15.123:0/1335191131 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cbe00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 0x7f43cc0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43d0a00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43cc0b2bb0 0x7f43cc162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 -- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43cc0b2bb0 msgr2=0x7f43cc162a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43cc0b2bb0 0x7f43cc162a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 -- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 msgr2=0x7f43cc0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 0x7f43cc0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 -- 172.21.15.123:0/1335191131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0cdec0 con 0x7f43cc1450d0 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cbe00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 0x7f43cc0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:18.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.352+0000 7f43cb400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43b8000c00 tx=0x7f43b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.353+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8037050 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.353+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8038040 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.353+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004450 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.353+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc161740 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.353+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc161d00 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.354+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc0cdc30 con 0x7f43cc1450d0 2024-04-25T19:03:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.355+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43b8030900 con 0x7f43cc1450d0 2024-04-25T19:03:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.356+0000 7f43c3e00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 0x7f43ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.357+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f43b80b6240 con 0x7f43cc1450d0 2024-04-25T19:03:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.357+0000 7f43cbe00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 0x7f43ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.357+0000 7f43cbe00640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 0x7f43ac0769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000f60 tx=0x7f43bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.361+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43b80805f0 con 0x7f43cc1450d0 2024-04-25T19:03:18.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.583+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43cc0b0640 con 0x7f43cc1450d0 2024-04-25T19:03:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.584+0000 7f43c3e00640 1 -- 172.21.15.123:0/1335191131 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f43b807fef0 con 0x7f43cc1450d0 2024-04-25T19:03:18.586 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:18.586 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:03:18.586 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.586+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 msgr2=0x7f43ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.586+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 0x7f43ac0769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000f60 tx=0x7f43bc009040 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.586+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 msgr2=0x7f43cc162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.586+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43b8000c00 tx=0x7f43b80046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 shutdown_connections 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f43ac0744b0 0x7f43ac0769c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43cc0b2bb0 0x7f43cc162a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc1450d0 0x7f43cc162500 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 --2- 172.21.15.123:0/1335191131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43cc0b0af0 0x7f43cc0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 >> 172.21.15.123:0/1335191131 conn(0x7f43cc073200 msgr2=0x7f43cc0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 shutdown_connections 2024-04-25T19:03:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.587+0000 7f43d1400640 1 -- 172.21.15.123:0/1335191131 wait complete. 2024-04-25T19:03:18.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:18.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.872+0000 7f90d2400640 1 Processor -- start 2024-04-25T19:03:18.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.872+0000 7f90d2400640 1 -- start start 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.874+0000 7f90d2400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90cc161b10 0x7f90cc161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.874+0000 7f90d2400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 0x7f90cc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.874+0000 7f90d2400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90cc071600 con 0x7f90cc0d1120 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.874+0000 7f90d2400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f90cc0d1530 con 0x7f90cc161b10 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.874+0000 7f90d2400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc162460 con 0x7f90cc154950 2024-04-25T19:03:18.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90cb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f90cc0d1120 0x7f90cc070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34480/0 (socket says 172.21.15.123:34480) 2024-04-25T19:03:18.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90cb400640 1 -- 172.21.15.123:0/2074808156 learned_addr learned my addr 172.21.15.123:0/2074808156 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:18.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90cbe00640 1 --2- 172.21.15.123:0/2074808156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 0x7f90cc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90c3e00640 1 --2- 172.21.15.123:0/2074808156 >> v2:172.21.15.146:3300/0 conn(0x7f90cc161b10 0x7f90cc161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90caa00640 1 -- 172.21.15.123:0/2074808156 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505550857 0 0) 0x7f90cc071600 con 0x7f90cc0d1120 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.875+0000 7f90caa00640 1 -- 172.21.15.123:0/2074808156 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90b0003520 con 0x7f90cc0d1120 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.876+0000 7f90cbe00640 1 -- 172.21.15.123:0/2074808156 >> v2:172.21.15.146:3300/0 conn(0x7f90cc161b10 msgr2=0x7f90cc161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.876+0000 7f90cbe00640 1 --2- 172.21.15.123:0/2074808156 >> v2:172.21.15.146:3300/0 conn(0x7f90cc161b10 0x7f90cc161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.876+0000 7f90cbe00640 1 -- 172.21.15.123:0/2074808156 >> v1:172.21.15.123:6789/0 conn(0x7f90cc0d1120 legacy=0x7f90cc070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.876+0000 7f90cbe00640 1 -- 172.21.15.123:0/2074808156 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc0aec30 con 0x7f90cc154950 2024-04-25T19:03:18.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.877+0000 7f90cbe00640 1 --2- 172.21.15.123:0/2074808156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 0x7f90cc154d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009d90 tx=0x7f90c402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e6a8126ab06d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.877+0000 7f90caa00640 1 -- 172.21.15.123:0/2074808156 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4007740 con 0x7f90cc154950 2024-04-25T19:03:18.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.877+0000 7f90caa00640 1 -- 172.21.15.123:0/2074808156 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c40078e0 con 0x7f90cc154950 2024-04-25T19:03:18.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.877+0000 7f90caa00640 1 -- 172.21.15.123:0/2074808156 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4007cb0 con 0x7f90cc154950 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 -- 172.21.15.123:0/2074808156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 msgr2=0x7f90cc154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 --2- 172.21.15.123:0/2074808156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 0x7f90cc154d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009d90 tx=0x7f90c402a530 comp rx=0 tx=0).stop 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 -- 172.21.15.123:0/2074808156 shutdown_connections 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 --2- 172.21.15.123:0/2074808156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc154950 0x7f90cc154d60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 --2- 172.21.15.123:0/2074808156 >> v2:172.21.15.146:3300/0 conn(0x7f90cc161b10 0x7f90cc161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 -- 172.21.15.123:0/2074808156 >> 172.21.15.123:0/2074808156 conn(0x7f90cc0d1ae0 msgr2=0x7f90cc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 -- 172.21.15.123:0/2074808156 shutdown_connections 2024-04-25T19:03:18.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.878+0000 7f90d2400640 1 -- 172.21.15.123:0/2074808156 wait complete. 2024-04-25T19:03:18.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.881+0000 7f90d2400640 1 Processor -- start 2024-04-25T19:03:18.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.881+0000 7f90d2400640 1 -- start start 2024-04-25T19:03:18.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc154950 0x7f90cc15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 0x7f90cc0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0af4d0 con 0x7f90cc0d1120 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0af1f0 con 0x7f90cc154950 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.882+0000 7f90d2400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc1552a0 con 0x7f90cc161b10 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49286/0 (socket says 172.21.15.123:49286) 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 -- 172.21.15.123:0/4022629716 learned_addr learned my addr 172.21.15.123:0/4022629716 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90c3e00640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc154950 0x7f90cc15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cbe00640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 0x7f90cc0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 -- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 msgr2=0x7f90cc0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 0x7f90cc0cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 -- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc154950 msgr2=0x7f90cc15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc154950 0x7f90cc15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 -- 172.21.15.123:0/4022629716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc1f8040 con 0x7f90cc0d1120 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cbe00640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 0x7f90cc0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.883+0000 7f90cb400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c390 tx=0x7f90bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.884+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc017070 con 0x7f90cc0d1120 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.884+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc005ce0 con 0x7f90cc0d1120 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.884+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0072f0 con 0x7f90cc0d1120 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.884+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc1f8320 con 0x7f90cc0d1120 2024-04-25T19:03:18.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.884+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc1f88e0 con 0x7f90cc0d1120 2024-04-25T19:03:18.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.885+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc076a40 con 0x7f90cc0d1120 2024-04-25T19:03:18.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.886+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f90bc01a070 con 0x7f90cc0d1120 2024-04-25T19:03:18.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.887+0000 7f90c3400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 0x7f90a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.887+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f90bc094990 con 0x7f90cc0d1120 2024-04-25T19:03:18.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.887+0000 7f90c3e00640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 0x7f90a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.888+0000 7f90c3e00640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 0x7f90a80769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0714a0 tx=0x7f90b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:18.892+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90bc05edd0 con 0x7f90cc0d1120 2024-04-25T19:03:19.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.100+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90cc070250 con 0x7f90cc0d1120 2024-04-25T19:03:19.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.101+0000 7f90c3400640 1 -- 172.21.15.123:0/4022629716 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18566 (secure 0 0 0) 0x7f90bc05e6d0 con 0x7f90cc0d1120 2024-04-25T19:03:19.107 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:19.107 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:02:44.672445+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:19.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 msgr2=0x7f90a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 0x7f90a80769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0714a0 tx=0x7f90b4008040 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 msgr2=0x7f90cc1e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c390 tx=0x7f90bc00c850 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 shutdown_connections 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f90a80744b0 0x7f90a80769c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90cc161b10 0x7f90cc0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90cc154950 0x7f90cc15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 --2- 172.21.15.123:0/4022629716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90cc0d1120 0x7f90cc1e87c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 >> 172.21.15.123:0/4022629716 conn(0x7f90cc0d1ae0 msgr2=0x7f90cc15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 shutdown_connections 2024-04-25T19:03:19.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.104+0000 7f90d2400640 1 -- 172.21.15.123:0/4022629716 wait complete. 2024-04-25T19:03:19.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:03:19.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.389+0000 7f8e21400640 1 Processor -- start 2024-04-25T19:03:19.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.389+0000 7f8e21400640 1 -- start start 2024-04-25T19:03:19.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.391+0000 7f8e21400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.391+0000 7f8e21400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.391+0000 7f8e21400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c0d2840 con 0x7f8e1c15ffe0 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.391+0000 7f8e21400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1c0d1800 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.391+0000 7f8e21400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c0ce880 con 0x7f8e1c0d2120 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e20a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e1be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e20a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:34388/0 (socket says 172.21.15.123:34388) 2024-04-25T19:03:19.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e1be00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:57634/0 (socket says 172.21.15.123:57634) 2024-04-25T19:03:19.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e20a00640 1 -- 172.21.15.123:0/3807829546 learned_addr learned my addr 172.21.15.123:0/3807829546 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:19.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e1be00640 1 -- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 msgr2=0x7f8e1c0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:19.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e1be00640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.392+0000 7f8e1be00640 1 -- 172.21.15.123:0/3807829546 >> v1:172.21.15.146:6789/0 conn(0x7f8e1c15ffe0 legacy=0x7f8e1c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:19.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.393+0000 7f8e1be00640 1 -- 172.21.15.123:0/3807829546 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c071fa0 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.393+0000 7f8e20a00640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:19.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.394+0000 7f8e1be00640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0097a0 tx=0x7f8e0c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d60554689b35cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:19.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.394+0000 7f8e1aa00640 1 -- 172.21.15.123:0/3807829546 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c037070 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e1aa00640 1 -- 172.21.15.123:0/3807829546 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c032440 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e1aa00640 1 -- 172.21.15.123:0/3807829546 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c032810 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 -- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0b2bb0 msgr2=0x7f8e1c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0097a0 tx=0x7f8e0c02a180 comp rx=0 tx=0).stop 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 -- 172.21.15.123:0/3807829546 shutdown_connections 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c0d2120 0x7f8e1c0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 --2- 172.21.15.123:0/3807829546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c06df10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 -- 172.21.15.123:0/3807829546 >> 172.21.15.123:0/3807829546 conn(0x7f8e1c0d36f0 msgr2=0x7f8e1c0d3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 -- 172.21.15.123:0/3807829546 shutdown_connections 2024-04-25T19:03:19.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.395+0000 7f8e21400640 1 -- 172.21.15.123:0/3807829546 wait complete. 2024-04-25T19:03:19.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.398+0000 7f8e21400640 1 Processor -- start 2024-04-25T19:03:19.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.398+0000 7f8e21400640 1 -- start start 2024-04-25T19:03:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c15ffe0 0x7f8e1c1eb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0c9a40 0x7f8e1c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c0ce510 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c072560 con 0x7f8e1c0c9a40 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e21400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c1603f0 con 0x7f8e1c15ffe0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.399+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49294/0 (socket says 172.21.15.123:49294) 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 -- 172.21.15.123:0/989167681 learned_addr learned my addr 172.21.15.123:0/989167681 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1b400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c15ffe0 0x7f8e1c1eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e20a00640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0c9a40 0x7f8e1c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 -- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c15ffe0 msgr2=0x7f8e1c1eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c15ffe0 0x7f8e1c1eb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 -- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0c9a40 msgr2=0x7f8e1c0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0c9a40 0x7f8e1c0c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 -- 172.21.15.123:0/989167681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c1ea780 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.400+0000 7f8e1be00640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098c0 tx=0x7f8e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.401+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c037040 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.401+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c030500 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.401+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c1eaa60 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.401+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0308d0 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.401+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c1eb020 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.402+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c050740 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.403+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e0c044020 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.404+0000 7f8e13e00640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 0x7f8df80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:19.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.404+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f8e0c0b58e0 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.404+0000 7f8e1b400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 0x7f8df80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:19.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.405+0000 7f8e1b400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 0x7f8df80769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e04007fd0 tx=0x7f8e04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:19.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.409+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e0c07fd00 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.647+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e1c0cdc90 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.648+0000 7f8e13e00640 1 -- 172.21.15.123:0/989167681 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f8e0c07f600 con 0x7f8e1c0b2bb0 2024-04-25T19:03:19.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:19.650 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:03:19.650 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 msgr2=0x7f8df80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 0x7f8df80769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e04007fd0 tx=0x7f8e04009040 comp rx=0 tx=0).stop 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 msgr2=0x7f8e1c159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0098c0 tx=0x7f8e0c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 shutdown_connections 2024-04-25T19:03:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8df80744b0 0x7f8df80769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e1c0c9a40 0x7f8e1c0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e1c15ffe0 0x7f8e1c1eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 --2- 172.21.15.123:0/989167681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e1c0b2bb0 0x7f8e1c159bd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.650+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 >> 172.21.15.123:0/989167681 conn(0x7f8e1c0d36f0 msgr2=0x7f8e1c163900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.651+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 shutdown_connections 2024-04-25T19:03:19.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:19.651+0000 7f8e21400640 1 -- 172.21.15.123:0/989167681 wait complete. 2024-04-25T19:03:20.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T19:02:39.551680+0000', 'modified': '2024-04-25T19:03:03.252379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6835/1141870368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 1141870368}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 1141870368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4251, 'qdb_cluster': [4251]} 2024-04-25T19:03:20.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T19:03:20.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T19:03:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.345+0000 7fa5a6000640 1 Processor -- start 2024-04-25T19:03:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.346+0000 7fa5a6000640 1 -- start start 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a6000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5a006d580 0x7fa5a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a6000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 0x7fa5a015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a6000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a006ded0 con 0x7fa5a006fe20 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a6000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a015f070 con 0x7fa5a006d580 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a6000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a006f340 con 0x7fa5a015e2f0 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a4c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa5a006fe20 0x7fa5a015b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34486/0 (socket says 172.21.15.123:34486) 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.348+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4099749492 learned_addr learned my addr 172.21.15.123:0/4099749492 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2040412017 0 0) 0x7fa590001110 con 0x7fa5a006fe20 2024-04-25T19:03:20.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa597e00640 1 --2- 172.21.15.123:0/4099749492 >> v2:172.21.15.146:3300/0 conn(0x7fa5a006d580 0x7fa5a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa580003520 con 0x7fa5a006fe20 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa5a5600640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 0x7fa5a015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 514186271 0 0) 0x7fa580003520 con 0x7fa5a006fe20 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 msgr2=0x7fa5a015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 0x7fa5a015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 >> v2:172.21.15.146:3300/0 conn(0x7fa5a006d580 msgr2=0x7fa5a006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 --2- 172.21.15.123:0/4099749492 >> v2:172.21.15.146:3300/0 conn(0x7fa5a006d580 0x7fa5a006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a006f620 con 0x7fa5a006fe20 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fa590001840 con 0x7fa5a006fe20 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 >> v1:172.21.15.123:6789/0 conn(0x7fa5a006fe20 legacy=0x7fa5a015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5800039b0 0x7fa580003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa580004800 0x7fa580004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 0x7fa580005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015f070 con 0x7fa5800039b0 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a006f340 con 0x7fa580004800 2024-04-25T19:03:20.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa58000baa0 con 0x7fa5a015e2f0 2024-04-25T19:03:20.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.349+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5800039b0 0x7fa580003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa597e00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa580004800 0x7fa580004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 msgr2=0x7fa580005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a015e2f0 0x7fa580005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa580004800 msgr2=0x7fa580004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa580004800 0x7fa580004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.350+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4099749492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa580003520 con 0x7fa5800039b0 2024-04-25T19:03:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.351+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5800039b0 0x7fa580003e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0070d80 tx=0x7fa580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e29d8dd42138020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.351+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590036070 con 0x7fa5800039b0 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.351+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590007e50 con 0x7fa5800039b0 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.351+0000 7fa59fe00640 1 -- 172.21.15.123:0/4099749492 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590005d30 con 0x7fa5800039b0 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 -- 172.21.15.123:0/4099749492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5800039b0 msgr2=0x7fa580003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 --2- 172.21.15.123:0/4099749492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5800039b0 0x7fa580003e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0070d80 tx=0x7fa580005830 comp rx=0 tx=0).stop 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a5600640 1 -- 172.21.15.123:0/4099749492 reap_dead start 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 -- 172.21.15.123:0/4099749492 shutdown_connections 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 -- 172.21.15.123:0/4099749492 >> 172.21.15.123:0/4099749492 conn(0x7fa5a0073200 msgr2=0x7fa5a0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 -- 172.21.15.123:0/4099749492 shutdown_connections 2024-04-25T19:03:20.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.352+0000 7fa5a6000640 1 -- 172.21.15.123:0/4099749492 wait complete. 2024-04-25T19:03:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.354+0000 7fa5a6000640 1 Processor -- start 2024-04-25T19:03:20.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.354+0000 7fa5a6000640 1 -- start start 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5800039b0 0x7fa5a0140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a013a840 0x7fa5a013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015e880 con 0x7fa5a013d850 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015e700 con 0x7fa5800039b0 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa5a6000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a015ea00 con 0x7fa5a013a840 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49312/0 (socket says 172.21.15.123:49312) 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 -- 172.21.15.123:0/2814004921 learned_addr learned my addr 172.21.15.123:0/2814004921 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 -- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a013a840 msgr2=0x7fa5a013ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a013a840 0x7fa5a013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 -- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5800039b0 msgr2=0x7fa5a0140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5800039b0 0x7fa5a0140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.356+0000 7fa597e00640 1 -- 172.21.15.123:0/2814004921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a013b190 con 0x7fa5a013d850 2024-04-25T19:03:20.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa597e00640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa588010860 tx=0x7fa588010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa59d600640 1 -- 172.21.15.123:0/2814004921 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58800f040 con 0x7fa5a013d850 2024-04-25T19:03:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa59d600640 1 -- 172.21.15.123:0/2814004921 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58800eae0 con 0x7fa5a013d850 2024-04-25T19:03:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa59d600640 1 -- 172.21.15.123:0/2814004921 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58801c400 con 0x7fa5a013d850 2024-04-25T19:03:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a0141820 con 0x7fa5a013d850 2024-04-25T19:03:20.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.357+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a0141de0 con 0x7fa5a013d850 2024-04-25T19:03:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.360+0000 7fa59d600640 1 -- 172.21.15.123:0/2814004921 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa58801c5a0 con 0x7fa5a013d850 2024-04-25T19:03:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.361+0000 7fa59d600640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 0x7fa57c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.362+0000 7fa59d600640 1 -- 172.21.15.123:0/2814004921 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa588095a60 con 0x7fa5a013d850 2024-04-25T19:03:20.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.362+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 0x7fa57c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.362+0000 7fa5ae5b6740 1 Processor -- start 2024-04-25T19:03:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.362+0000 7fa5ae5b6740 1 -- start start 2024-04-25T19:03:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.363+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 0x7fa57c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500922590 0x55b500d2f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b500c49600 con 0x55b50091a3b0 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b500be6100 con 0x55b500922590 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55b500c455c0 con 0x55b500943270 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49316/0 (socket says 172.21.15.123:49316) 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 -- 172.21.15.123:0/2351727323 learned_addr learned my addr 172.21.15.123:0/2351727323 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 -- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 msgr2=0x55b500d14620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa5a5600640 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500922590 0x55b500d2f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d14620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 -- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500922590 msgr2=0x55b500d2f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500922590 0x55b500d2f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.364+0000 7fa597e00640 1 -- 172.21.15.123:0/2351727323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50095b2d0 con 0x55b50091a3b0 2024-04-25T19:03:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.365+0000 7fa597e00640 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5880090e0 tx=0x7fa58805be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1f7e901503a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.365+0000 7fa595200640 1 -- 172.21.15.123:0/2351727323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588021070 con 0x55b50091a3b0 2024-04-25T19:03:20.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.365+0000 7fa595200640 1 -- 172.21.15.123:0/2351727323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5880660e0 con 0x55b50091a3b0 2024-04-25T19:03:20.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.365+0000 7fa595200640 1 -- 172.21.15.123:0/2351727323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588074dd0 con 0x55b50091a3b0 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/2351727323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 msgr2=0x55b500d2f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5880090e0 tx=0x7fa58805be60 comp rx=0 tx=0).stop 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/2351727323 shutdown_connections 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500922590 0x55b500d2f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500d2f350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/2351727323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/2351727323 >> 172.21.15.123:0/2351727323 conn(0x55b5009373c0 msgr2=0x55b500ce9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/2351727323 shutdown_connections 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.366+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/2351727323 wait complete. 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.367+0000 7fa5ae5b6740 1 -- start start 2024-04-25T19:03:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.367+0000 7fa5ae5b6740 1 Processor -- start 2024-04-25T19:03:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b50091a3b0 0x55b500d2a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b500bd0f70 con 0x55b500922590 2024-04-25T19:03:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b500d2e1d0 con 0x55b50091a3b0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55b500d2aec0 con 0x55b500943270 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5ae5b6740 10 client.0 _ll_register_callbacks cb 0x55b500c3f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49330/0 (socket says 172.21.15.123:49330) 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 -- 172.21.15.123:0/3056136514 learned_addr learned my addr 172.21.15.123:0/3056136514 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa5a5600640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 -- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 msgr2=0x55b500d2bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d2bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 -- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b50091a3b0 msgr2=0x55b500d2a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b50091a3b0 0x55b500d2a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.369+0000 7fa597e00640 1 -- 172.21.15.123:0/3056136514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b500a256b0 con 0x55b500922590 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa597e00640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa58805be00 tx=0x7fa588010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4a64a23500fe97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588063bc0 con 0x55b500922590 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa58801bc10 con 0x55b500922590 2024-04-25T19:03:20.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5880660e0 con 0x55b500922590 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa5880685e0 con 0x55b500922590 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 10 client.4809 fetch_fsmap learned FSMap version 9 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b500d3e220 con 0x55b500922590 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5880660e0 con 0x55b500922590 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fa588060b40 con 0x55b500922590 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 10 client.4809 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 10 client.4809 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 20 client.4809 populate_metadata read hostname 'smithi123' 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500901e40 0x55b500d421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 4 client.4809 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.370+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 --> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b500c57210 con 0x55b500901e40 2024-04-25T19:03:20.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.371+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500901e40 0x55b500d421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.372+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500901e40 0x55b500d421a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a30f6b1c064d262 server_cookie=d9e4a8fafe94b98c in_seq=0 out_seq=0 2024-04-25T19:03:20.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.372+0000 7fa58d600640 10 client.4809 ms_handle_connect on v2:172.21.15.146:6834/1141870368 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.374+0000 7fa58d600640 1 -- 172.21.15.123:0/3056136514 <== mds.0 v2:172.21.15.146:6834/1141870368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b500c57210 con 0x55b500901e40 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.374+0000 7fa58d600640 10 client.4809 handle_command_reply: tid=0 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.374+0000 7fa5ae5b6740 1 client.4809 shutdown 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 msgr2=0x55b500d2b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa58805be00 tx=0x7fa588010f70 comp rx=0 tx=0).stop 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 shutdown_connections 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500901e40 0x55b500d421a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500943270 0x55b500d2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d2b8a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa58d600640 0 client.4809 ms_handle_reset on v2:172.21.15.146:6834/1141870368 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/3056136514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b50091a3b0 0x55b500d2a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 >> 172.21.15.123:0/3056136514 conn(0x55b5009373c0 msgr2=0x55b500ce9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 shutdown_connections 2024-04-25T19:03:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/3056136514 wait complete. 2024-04-25T19:03:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.375+0000 7fa5ae5b6740 20 client.4809 trim_cache size 0 max 16384 2024-04-25T19:03:20.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.395+0000 7fa5ae5b6740 1 Processor -- start 2024-04-25T19:03:20.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.395+0000 7fa5ae5b6740 1 -- start start 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b50091a3b0 0x55b500d52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01ff3f0 con 0x55b500922590 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01f60d0 con 0x55b500943270 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00b6610 con 0x55b50091a3b0 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49344/0 (socket says 172.21.15.123:49344) 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 -- 172.21.15.123:0/522462427 learned_addr learned my addr 172.21.15.123:0/522462427 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 -- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b50091a3b0 msgr2=0x55b500d52480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b50091a3b0 0x55b500d52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 -- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 msgr2=0x55b500d2a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 -- 172.21.15.123:0/522462427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b500d2dd00 con 0x55b500922590 2024-04-25T19:03:20.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.397+0000 7fa597e00640 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa588095d80 tx=0x7fa588061000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a24d8e589d1f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa58fe00640 1 -- 172.21.15.123:0/522462427 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588074de0 con 0x55b500922590 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa58fe00640 1 -- 172.21.15.123:0/522462427 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5880660e0 con 0x55b500922590 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa58fe00640 1 -- 172.21.15.123:0/522462427 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58805be60 con 0x55b500922590 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/522462427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 msgr2=0x55b500d29db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa588095d80 tx=0x7fa588061000 comp rx=0 tx=0).stop 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/522462427 shutdown_connections 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b500922590 0x55b500d29db0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/522462427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b50091a3b0 0x55b500d52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.398+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/522462427 >> 172.21.15.123:0/522462427 conn(0x55b5009373c0 msgr2=0x55b500d2b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.399+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/522462427 shutdown_connections 2024-04-25T19:03:20.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.399+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/522462427 wait complete. 2024-04-25T19:03:20.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.399+0000 7fa5ae5b6740 1 -- start start 2024-04-25T19:03:20.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.400+0000 7fa5ae5b6740 1 Processor -- start 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500922590 0x55b500db0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d31650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b500bd0f70 con 0x55b50091a3b0 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b500d2e030 con 0x55b500943270 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55b500d168d0 con 0x55b500922590 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5ae5b6740 10 client.0 _ll_register_callbacks cb 0x7fa58805f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5a4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5a4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49348/0 (socket says 172.21.15.123:49348) 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.401+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4100962586 learned_addr learned my addr 172.21.15.123:0/4100962586 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500922590 msgr2=0x55b500db0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa597e00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500922590 0x55b500db0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500922590 0x55b500db0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a5600640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d31650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 msgr2=0x55b500d31650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d31650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 -- 172.21.15.123:0/4100962586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b500ce83f0 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5a4c00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c3e0 tx=0x7fa5a006f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1691f03eb4200804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa58d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590007480 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa590007620 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.402+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5880660e0 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa59005b050 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 10 client.4815 fetch_fsmap learned FSMap version 9 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b500db0b00 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5880660e0 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fa59005e020 con 0x55b50091a3b0 2024-04-25T19:03:20.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 10 client.4815 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T19:03:20.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 10 client.4815 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:03:20.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 20 client.4815 populate_metadata read hostname 'smithi123' 2024-04-25T19:03:20.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500cea470 0x55b500d50fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:20.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 4 client.4815 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:20.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.403+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 --> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] -- command(tid 0: {"prefix": "status"}) -- 0x55b500c51080 con 0x55b500cea470 2024-04-25T19:03:20.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.404+0000 7fa597e00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500cea470 0x55b500d50fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:20.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.405+0000 7fa597e00640 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500cea470 0x55b500d50fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496b7c05b989fbff server_cookie=48036ae510b165a9 in_seq=0 out_seq=0 2024-04-25T19:03:20.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.405+0000 7fa58d600640 10 client.4815 ms_handle_connect on v2:172.21.15.146:6834/1141870368 2024-04-25T19:03:20.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.405+0000 7fa58d600640 1 -- 172.21.15.123:0/4100962586 <== mds.0 v2:172.21.15.146:6834/1141870368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55b500c51080 con 0x55b500cea470 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.405+0000 7fa58d600640 10 client.4815 handle_command_reply: tid=0 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.405+0000 7fa5ae5b6740 1 client.4815 shutdown 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 msgr2=0x55b500dac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c3e0 tx=0x7fa5a006f7f0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 shutdown_connections 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:6834/1141870368,v1:172.21.15.146:6835/1141870368] conn(0x55b500cea470 0x55b500d50fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b500943270 0x55b500d31650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55b500922590 0x55b500db0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa58d600640 0 client.4815 ms_handle_reset on v2:172.21.15.146:6834/1141870368 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 --2- 172.21.15.123:0/4100962586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50091a3b0 0x55b500dac790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 >> 172.21.15.123:0/4100962586 conn(0x55b5009373c0 msgr2=0x55b500d2b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 shutdown_connections 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 1 -- 172.21.15.123:0/4100962586 wait complete. 2024-04-25T19:03:20.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5ae5b6740 20 client.4815 trim_cache size 0 max 16384 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "a2d72801-e932-4284-9776-fce727cd5af1", 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4251, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 40.346882342000001, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 9, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 28, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 45.946108983000002 2024-04-25T19:03:20.410 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 msgr2=0x7fa57c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 0x7fa57c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590007000 comp rx=0 tx=0).stop 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 msgr2=0x7fa5a0139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.406+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa588010860 tx=0x7fa588010d20 comp rx=0 tx=0).stop 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 shutdown_connections 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa57c0744b0 0x7fa57c0769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5a013a840 0x7fa5a013ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a013d850 0x7fa5a0139920 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 --2- 172.21.15.123:0/2814004921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5800039b0 0x7fa5a0140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:20.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 >> 172.21.15.123:0/2814004921 conn(0x7fa5a0073200 msgr2=0x7fa5a0160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:20.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 shutdown_connections 2024-04-25T19:03:20.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:20.407+0000 7fa5a6000640 1 -- 172.21.15.123:0/2814004921 wait complete. 2024-04-25T19:03:20.435 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T19:03:20.448 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T19:03:20.448 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T19:03:20.448 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T19:03:20.449 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T19:03:20.460 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T19:03:20.460 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T19:03:20.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.460 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:03:20.510 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T19:03:20.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T19:03:20.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T19:03:20.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.699 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:03:20.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T19:03:20.865 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-04-25T19:03:20.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:03:20.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.869 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:03:20.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:20.927 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T19:03:21.011 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T19:03:21.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:03:21.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:21.015 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:03:21.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:21.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T19:03:21.150 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T19:03:21.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:03:21.155 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T19:03:21.155 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:03:21.155 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:21.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:03:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.437+0000 7f47df200640 1 Processor -- start 2024-04-25T19:03:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.437+0000 7f47df200640 1 -- start start 2024-04-25T19:03:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.439+0000 7f47df200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f47e0161ad0 0x7f47e0161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47df200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e0143840 con 0x7f47e006d580 2024-04-25T19:03:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47df200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e0143b20 con 0x7f47e00d04f0 2024-04-25T19:03:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47df200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f47e00b60b0 con 0x7f47e0161ad0 2024-04-25T19:03:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47de800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f47e006d580 0x7f47e006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:34498/0 (socket says 172.21.15.123:34498) 2024-04-25T19:03:21.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47de800640 1 -- 172.21.15.123:0/1570466971 learned_addr learned my addr 172.21.15.123:0/1570466971 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.440+0000 7f47d7e00640 1 --2- 172.21.15.123:0/1570466971 >> v2:172.21.15.146:3300/0 conn(0x7f47e0161ad0 0x7f47e0161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 762764306 0 0) 0x7f47e0143840 con 0x7f47e006d580 2024-04-25T19:03:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47c4003520 con 0x7f47e006d580 2024-04-25T19:03:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 495239077 0 0) 0x7f47c4003520 con 0x7f47e006d580 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 >> v2:172.21.15.146:3300/0 conn(0x7f47e0161ad0 msgr2=0x7f47e0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 --2- 172.21.15.123:0/1570466971 >> v2:172.21.15.146:3300/0 conn(0x7f47e0161ad0 0x7f47e0161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 >> v1:172.21.15.146:6789/0 conn(0x7f47e00d04f0 legacy=0x7f47e00cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e0143e00 con 0x7f47e006d580 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f47d80019f0 con 0x7f47e006d580 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 >> v1:172.21.15.123:6789/0 conn(0x7f47e006d580 legacy=0x7f47e006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4004800 0x7f47c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47c40052a0 0x7f47c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e00b60b0 con 0x7f47c4003a00 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47c400c230 con 0x7f47c4004800 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.441+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47c400c430 con 0x7f47c40052a0 2024-04-25T19:03:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47d7e00640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4004800 0x7f47c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47de800640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47c40052a0 0x7f47c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 -- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47c40052a0 msgr2=0x7f47c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47c40052a0 0x7f47c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 -- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4004800 msgr2=0x7f47c4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4004800 0x7f47c4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47dde00640 1 -- 172.21.15.123:0/1570466971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c4003520 con 0x7f47c4003a00 2024-04-25T19:03:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.442+0000 7f47de800640 1 -- 172.21.15.123:0/1570466971 reap_dead start 2024-04-25T19:03:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.443+0000 7f47dde00640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47c4003e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47d0001750 tx=0x7f47c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f465d5aac1e060b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.444+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0001370 con 0x7f47c4003a00 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.444+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d00052d0 con 0x7f47c4003a00 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.444+0000 7f47dd400640 1 -- 172.21.15.123:0/1570466971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d002abc0 con 0x7f47c4003a00 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.444+0000 7f47df200640 1 -- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 msgr2=0x7f47c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.444+0000 7f47df200640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47c4003e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47d0001750 tx=0x7f47c4005830 comp rx=0 tx=0).stop 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.445+0000 7f47df200640 1 -- 172.21.15.123:0/1570466971 shutdown_connections 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.445+0000 7f47df200640 1 --2- 172.21.15.123:0/1570466971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47c4003e70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.445+0000 7f47df200640 1 -- 172.21.15.123:0/1570466971 >> 172.21.15.123:0/1570466971 conn(0x7f47e00ce090 msgr2=0x7f47e00ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.445+0000 7f47df200640 1 -- 172.21.15.123:0/1570466971 shutdown_connections 2024-04-25T19:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.445+0000 7f47df200640 1 -- 172.21.15.123:0/1570466971 wait complete. 2024-04-25T19:03:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.448+0000 7f47df200640 1 Processor -- start 2024-04-25T19:03:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.448+0000 7f47df200640 1 -- start start 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47e015d350 0x7f47e01450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 0x7f47e01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0143460 con 0x7f47c4003a00 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0162750 con 0x7f47e015d350 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.450+0000 7f47df200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47e01435e0 con 0x7f47e01603e0 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49360/0 (socket says 172.21.15.123:49360) 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 -- 172.21.15.123:0/2234753938 learned_addr learned my addr 172.21.15.123:0/2234753938 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47de800640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 0x7f47e01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47d7e00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47e015d350 0x7f47e01450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 -- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 msgr2=0x7f47e01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 0x7f47e01607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 -- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47e015d350 msgr2=0x7f47e01450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47e015d350 0x7f47e01450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47dde00640 1 -- 172.21.15.123:0/2234753938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e015f340 con 0x7f47c4003a00 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.451+0000 7f47de800640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 0x7f47e01607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47dde00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f47d0000c00 tx=0x7f47d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d003f030 con 0x7f47c4003a00 2024-04-25T19:03:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d0005480 con 0x7f47c4003a00 2024-04-25T19:03:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0037040 con 0x7f47c4003a00 2024-04-25T19:03:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e015f620 con 0x7f47c4003a00 2024-04-25T19:03:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.452+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e015fbe0 con 0x7f47c4003a00 2024-04-25T19:03:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.453+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e0050740 con 0x7f47c4003a00 2024-04-25T19:03:21.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.454+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47d0042070 con 0x7f47c4003a00 2024-04-25T19:03:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.455+0000 7f47d6000640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 0x7f47bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.456+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f47d00b4c30 con 0x7f47c4003a00 2024-04-25T19:03:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.456+0000 7f47d7e00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 0x7f47bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:21.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.457+0000 7f47d7e00640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 0x7f47bc0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f47e015d110 tx=0x7f47c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:21.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.460+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47d007f050 con 0x7f47c4003a00 2024-04-25T19:03:21.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.701+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f47e01616b0 con 0x7f47c4003a00 2024-04-25T19:03:21.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.702+0000 7f47d6000640 1 -- 172.21.15.123:0/2234753938 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f47d007e950 con 0x7f47c4003a00 2024-04-25T19:03:21.703 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T19:03:21.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 msgr2=0x7f47bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 0x7f47bc0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f47e015d110 tx=0x7f47c8008040 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 msgr2=0x7f47e013dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f47d0000c00 tx=0x7f47d0001120 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 shutdown_connections 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47bc0744b0 0x7f47bc0769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47e01603e0 0x7f47e01607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47e015d350 0x7f47e01450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 --2- 172.21.15.123:0/2234753938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47c4003a00 0x7f47e013dda0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.704+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 >> 172.21.15.123:0/2234753938 conn(0x7f47e00ce090 msgr2=0x7f47e0142910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.705+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 shutdown_connections 2024-04-25T19:03:21.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21.705+0000 7f47df200640 1 -- 172.21.15.123:0/2234753938 wait complete. 2024-04-25T19:03:21.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:03:21.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:03:21.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T19:03:21.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T19:03:21.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:21.735 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:03:21.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:03:21.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:03:21.735 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:03:21.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:21.735 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T19:03:21.783 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:21.783 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:21.784 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:03:21.785 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42596sec preferred_lft 42596sec 2024-04-25T19:03:21.785 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T19:03:21.785 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:21.785 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T19:03:21.786 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:21.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:22.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:22.021 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:03:22.086 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T19:03:22.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:22.090 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T19:03:22.134 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T19:03:22.134 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:03:22.134 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:03:22.134 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:03:22.134 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:03:22.135 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:22.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:22.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:22.392 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:03:22.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:22.440 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T19:03:22.488 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:22.488 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:22.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:03:22.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T19:03:22.488 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:22.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:22.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:22.668 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T19:03:22.668 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:22.668 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:22.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:22.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:03:22.863 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:03:22.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:23.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:23.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:23.126 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:23.167 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:03:23.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:23.167 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:23.259 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T19:03:23.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:23.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:23.460 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:23.460 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:24.220 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:24.220 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:24.220 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T19:03:24.220 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 0@a2d72801-e932-4284-9776-fce727cd5af1.cephfs=/ 2024-04-25T19:03:24.220 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.123:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T19:03:24.223 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T19:03:24.223 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:03:24.223 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:03:24.223 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:03:24.223 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:03:24.228 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:03:24.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:24.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:24.299 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:24.405 INFO:teuthology.orchestra.run.smithi123.stdout:client4478 2024-04-25T19:03:24.406 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:24.406 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:24.451 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:24.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:24.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:24.600 INFO:teuthology.orchestra.run.smithi123.stdout:client4478 2024-04-25T19:03:24.601 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:24.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:24.709 INFO:teuthology.orchestra.run.smithi123.stdout:client4478 2024-04-25T19:03:24.709 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a2d72801-e932-4284-9776-fce727cd5af1.client4478/status') 2024-04-25T19:03:24.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4478 (3)172.21.15.123:0/4180339873 2024-04-25T19:03:24.824 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-04-25T19:03:24.824 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-04-25T19:03:24.825 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-04-25T19:03:24.825 INFO:teuthology.orchestra.run.smithi123.stderr:71 bytes copied, 0.000119498 s, 594 kB/s 2024-04-25T19:03:24.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:24.836 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T19:03:24.836 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:03:24.836 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T19:03:24.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:03:25.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.081+0000 7f0429200640 1 Processor -- start 2024-04-25T19:03:25.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.081+0000 7f0429200640 1 -- start start 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0429200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f042415a810 0x7f04240b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0429200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0429200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0424161200 con 0x7f0424160df0 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0429200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f042406f130 con 0x7f042415a810 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0429200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f042415ac20 con 0x7f04240b4b80 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0423400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f042415a810 0x7f04240b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0423400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f042415a810 0x7f04240b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49368/0 (socket says 172.21.15.123:49368) 2024-04-25T19:03:25.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.083+0000 7f0423400640 1 -- 172.21.15.123:0/640610083 learned_addr learned my addr 172.21.15.123:0/640610083 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.084+0000 7f0423400640 1 -- 172.21.15.123:0/640610083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 msgr2=0x7f04241608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.084+0000 7f041aa00640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.084+0000 7f0423400640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.084+0000 7f0423400640 1 -- 172.21.15.123:0/640610083 >> v1:172.21.15.146:6789/0 conn(0x7f0424160df0 legacy=0x7f042415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.084+0000 7f0423400640 1 -- 172.21.15.123:0/640610083 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04240b7060 con 0x7f042415a810 2024-04-25T19:03:25.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.085+0000 7f0423400640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240b4640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0414009870 tx=0x7f041402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c6dd77d0d2df33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:25.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.085+0000 7f0422a00640 1 -- 172.21.15.123:0/640610083 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414037070 con 0x7f042415a810 2024-04-25T19:03:25.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.085+0000 7f0422a00640 1 -- 172.21.15.123:0/640610083 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0414032440 con 0x7f042415a810 2024-04-25T19:03:25.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.085+0000 7f0422a00640 1 -- 172.21.15.123:0/640610083 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414032810 con 0x7f042415a810 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 -- 172.21.15.123:0/640610083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 msgr2=0x7f04240b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240b4640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0414009870 tx=0x7f041402a180 comp rx=0 tx=0).stop 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 -- 172.21.15.123:0/640610083 shutdown_connections 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 --2- 172.21.15.123:0/640610083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240b4640 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 -- 172.21.15.123:0/640610083 >> 172.21.15.123:0/640610083 conn(0x7f04240ce090 msgr2=0x7f04240ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 -- 172.21.15.123:0/640610083 shutdown_connections 2024-04-25T19:03:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.086+0000 7f0429200640 1 -- 172.21.15.123:0/640610083 wait complete. 2024-04-25T19:03:25.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.088+0000 7f0429200640 1 Processor -- start 2024-04-25T19:03:25.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.089+0000 7f0429200640 1 -- start start 2024-04-25T19:03:25.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 0x7f04240af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04240cf6f0 con 0x7f042415a810 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f04240b7620 con 0x7f0424160df0 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f0429200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f04240cf9d0 con 0x7f04240b4b80 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f041aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f041aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49380/0 (socket says 172.21.15.123:49380) 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.090+0000 7f041aa00640 1 -- 172.21.15.123:0/882608604 learned_addr learned my addr 172.21.15.123:0/882608604 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f0423400640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f0423e00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 0x7f04240af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 -- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 msgr2=0x7f04241444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 -- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 msgr2=0x7f04240af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:25.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 0x7f04240af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 -- 172.21.15.123:0/882608604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424144fc0 con 0x7f042415a810 2024-04-25T19:03:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f0423e00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 0x7f04240af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041aa00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c7f0 tx=0x7f040c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:25.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c015070 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c004220 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.091+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c007620 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.092+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04240ada40 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.092+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04240ae000 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.093+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0424050740 con 0x7f042415a810 2024-04-25T19:03:25.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.094+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f040c0079d0 con 0x7f042415a810 2024-04-25T19:03:25.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.095+0000 7f041be00640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 0x7f04000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:25.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.095+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f040c095990 con 0x7f042415a810 2024-04-25T19:03:25.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.095+0000 7f0423400640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 0x7f04000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:25.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.096+0000 7f0423400640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 0x7f04000769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0414034000 tx=0x7f041402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:25.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.100+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0424144fc0 con 0x7f042415a810 2024-04-25T19:03:25.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.339+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04240ad7d0 con 0x7f042415a810 2024-04-25T19:03:25.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.340+0000 7f041be00640 1 -- 172.21.15.123:0/882608604 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f040c05fda0 con 0x7f042415a810 2024-04-25T19:03:25.342 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 msgr2=0x7f04000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 0x7f04000769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0414034000 tx=0x7f041402e000 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 msgr2=0x7f04240ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c7f0 tx=0x7f040c00ccb0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 shutdown_connections 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f04000744b0 0x7f04000769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0424160df0 0x7f04240af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f042415a810 0x7f04240ace30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 --2- 172.21.15.123:0/882608604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04240b4b80 0x7f04241444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 >> 172.21.15.123:0/882608604 conn(0x7f04240ce090 msgr2=0x7f04240b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 shutdown_connections 2024-04-25T19:03:25.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25.343+0000 7f0429200640 1 -- 172.21.15.123:0/882608604 wait complete. 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:03:25.368 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T19:03:25.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:25.369 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T19:03:25.413 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:25.413 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:25.413 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:25.414 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42592sec preferred_lft 42592sec 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 62:a3:19:86:2b:d1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::942b:40ff:fe0f:8567/64 scope link 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 62:a3:19:86:2b:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::60a3:19ff:fe86:2bd1/64 scope link 2024-04-25T19:03:25.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:25.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:25.416 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:03:25.536 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T19:03:25.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:25.540 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T19:03:25.590 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T19:03:25.591 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:03:25.591 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:03:25.591 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:03:25.591 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:03:25.591 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:25.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:25.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:25.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:25.802 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:03:25.849 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:03:25.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:25.851 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-25T19:03:25.898 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T19:03:25.899 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:25.899 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:25.899 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T19:03:25.899 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T19:03:25.899 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:26.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:26.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:26.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:26.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T19:03:26.154 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 1a:c7:9b:c1:48:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::18c7:9bff:fec1:48c2/64 scope link 2024-04-25T19:03:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:26.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T19:03:26.241 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 1a:c7:9b:c1:48:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::18c7:9bff:fec1:48c2/64 scope link 2024-04-25T19:03:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:26.245 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T19:03:26.246 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:26.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:26.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:26.519 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:26.519 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:03:26.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2024-04-25T19:03:26.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T19:03:26.520 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:03:26.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:26.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:26.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:26.726 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T19:03:26.768 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T19:03:26.768 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:26.768 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T19:03:26.860 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T19:03:26.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:26.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:27.063 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:27.063 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:27.108 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:27.108 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:27.108 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T19:03:27.108 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 2@a2d72801-e932-4284-9776-fce727cd5af1.cephfs=/ 2024-04-25T19:03:27.108 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.123:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T19:03:27.110 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T19:03:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:03:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:03:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:03:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:03:27.116 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:03:27.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:27.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T19:03:27.189 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:27.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:27.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:27.308 INFO:teuthology.orchestra.run.smithi123.stdout:client4588 2024-04-25T19:03:27.309 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:27.309 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:27.360 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:27.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:27.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:27.478 INFO:teuthology.orchestra.run.smithi123.stdout:client4588 2024-04-25T19:03:27.478 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:27.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:27.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:27.595 INFO:teuthology.orchestra.run.smithi123.stdout:client4588 2024-04-25T19:03:27.596 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a2d72801-e932-4284-9776-fce727cd5af1.client4588/status') 2024-04-25T19:03:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:27.729 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4588 (3)172.21.15.123:0/1233065718 2024-04-25T19:03:27.729 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-04-25T19:03:27.729 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-04-25T19:03:27.729 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-04-25T19:03:27.729 INFO:teuthology.orchestra.run.smithi123.stderr:71 bytes copied, 0.00010951 s, 648 kB/s 2024-04-25T19:03:27.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:27.742 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T19:03:27.742 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:03:27.742 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T19:03:27.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:03:28.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.009+0000 7fcbc6a00640 1 Processor -- start 2024-04-25T19:03:28.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.010+0000 7fcbc6a00640 1 -- start start 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcbc0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbc6a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc00aec30 con 0x7fcbc0161b10 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbc6a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc00aef10 con 0x7fcbc0154950 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbc6a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00d1530 con 0x7fcbc00d1120 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbb7400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcbc0161b10 0x7fcbc0161f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:50462/0 (socket says 172.21.15.123:50462) 2024-04-25T19:03:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.012+0000 7fcbb7400640 1 -- 172.21.15.123:0/2540156778 learned_addr learned my addr 172.21.15.123:0/2540156778 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbfe00640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcbc0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139326698 0 0) 0x7fcbc00aec30 con 0x7fcbc0161b10 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcba4003520 con 0x7fcbc0161b10 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 918552048 0 0) 0x7fcba4003520 con 0x7fcbc0161b10 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 msgr2=0x7fcbc0070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcbc0070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 >> v1:172.21.15.146:6789/0 conn(0x7fcbc0154950 legacy=0x7fcbc0154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc00af1f0 con 0x7fcbc0161b10 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fcba8001670 con 0x7fcbc0161b10 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.013+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 >> v1:172.21.15.123:6789/0 conn(0x7fcbc0161b10 legacy=0x7fcbc0161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcba40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba4003f60 0x7fcba4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4004d40 0x7fcba40051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00d1530 con 0x7fcba4003f60 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcba400bc80 con 0x7fcba4004d40 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcba400be80 con 0x7fcbc00d1120 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbb7400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba4003f60 0x7fcba4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbbfe00640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcba40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbb7400640 1 -- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 msgr2=0x7fcba40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbb7400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc00d1120 0x7fcba40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbb7400640 1 -- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4004d40 msgr2=0x7fcba40051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.014+0000 7fcbc4a00640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4004d40 0x7fcba40051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.015+0000 7fcbb7400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4004d40 0x7fcba40051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.015+0000 7fcbb7400640 1 -- 172.21.15.123:0/2540156778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba4003520 con 0x7fcba4003f60 2024-04-25T19:03:28.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.015+0000 7fcbc4a00640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4004d40 0x7fcba40051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.016+0000 7fcbb7400640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba4003f60 0x7fcba4004800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00b1260 tx=0x7fcba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f809f53acb89a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.016+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007d30 con 0x7fcba4003f60 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.016+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba8005ad0 con 0x7fcba4003f60 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.016+0000 7fcbbf400640 1 -- 172.21.15.123:0/2540156778 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007070 con 0x7fcba4003f60 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2540156778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba4003f60 msgr2=0x7fcba4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2540156778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba4003f60 0x7fcba4004800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00b1260 tx=0x7fcba4005830 comp rx=0 tx=0).stop 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc4a00640 1 -- 172.21.15.123:0/2540156778 reap_dead start 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2540156778 shutdown_connections 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2540156778 >> 172.21.15.123:0/2540156778 conn(0x7fcbc00d1ae0 msgr2=0x7fcbc00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2540156778 shutdown_connections 2024-04-25T19:03:28.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.017+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2540156778 wait complete. 2024-04-25T19:03:28.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.019+0000 7fcbc6a00640 1 Processor -- start 2024-04-25T19:03:28.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.020+0000 7fcbc6a00640 1 -- start start 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4003f60 0x7fcbc01472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc009beb0 0x7fcbc01ea360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00ae960 con 0x7fcbc009d310 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00ae7e0 con 0x7fcba4003f60 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc6a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc006d8f0 con 0x7fcbc009beb0 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.021+0000 7fcbc4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42892/0 (socket says 172.21.15.123:42892) 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 -- 172.21.15.123:0/2957837107 learned_addr learned my addr 172.21.15.123:0/2957837107 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbb7400640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc009beb0 0x7fcbc01ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbbfe00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4003f60 0x7fcbc01472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 -- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc009beb0 msgr2=0x7fcbc01ea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc009beb0 0x7fcbc01ea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 -- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4003f60 msgr2=0x7fcbc01472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4003f60 0x7fcbc01472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 -- 172.21.15.123:0/2957837107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc01f4490 con 0x7fcbc009d310 2024-04-25T19:03:28.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.022+0000 7fcbc4a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0161a70 tx=0x7fcbb800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.023+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8006d50 con 0x7fcbc009d310 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.023+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb8003970 con 0x7fcbc009d310 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.023+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8003da0 con 0x7fcbc009d310 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.023+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc009c620 con 0x7fcbc009d310 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.023+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01e7ad0 con 0x7fcbc009d310 2024-04-25T19:03:28.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.024+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc009cb40 con 0x7fcbc009d310 2024-04-25T19:03:28.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.025+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcbb801b020 con 0x7fcbc009d310 2024-04-25T19:03:28.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.026+0000 7fcbbcc00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:28.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.026+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fcbb8095300 con 0x7fcbc009d310 2024-04-25T19:03:28.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.027+0000 7fcbbfe00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:28.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.027+0000 7fcbbfe00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009a10 tx=0x7fcbb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:28.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.031+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbc009cb40 con 0x7fcbc009d310 2024-04-25T19:03:28.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.279+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcbc01f4660 con 0x7fcbc009d310 2024-04-25T19:03:28.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.280+0000 7fcbbcc00640 1 -- 172.21.15.123:0/2957837107 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fcbb805f680 con 0x7fcbc009d310 2024-04-25T19:03:28.282 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 msgr2=0x7fcb9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009a10 tx=0x7fcbb0008040 comp rx=0 tx=0).stop 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 msgr2=0x7fcbc009d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0161a70 tx=0x7fcbb800dc00 comp rx=0 tx=0).stop 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 shutdown_connections 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc009d310 0x7fcbc009d720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc009beb0 0x7fcbc01ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 --2- 172.21.15.123:0/2957837107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4003f60 0x7fcbc01472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 >> 172.21.15.123:0/2957837107 conn(0x7fcbc00d1ae0 msgr2=0x7fcbc015c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 shutdown_connections 2024-04-25T19:03:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:28.283+0000 7fcbc6a00640 1 -- 172.21.15.123:0/2957837107 wait complete. 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:03:28.314 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T19:03:28.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:28.314 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-04-25T19:03:28.369 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:28.369 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:28.369 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:03:28.370 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:28.371 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:03:28.371 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42609sec preferred_lft 42609sec 2024-04-25T19:03:28.371 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-04-25T19:03:28.371 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:28.372 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T19:03:28.372 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:28.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:28.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:28.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:28.839 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:03:28.902 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T19:03:28.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:28.906 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-04-25T19:03:28.948 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-04-25T19:03:28.948 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:03:28.948 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:03:28.948 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:03:28.949 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:03:28.949 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:29.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:29.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:29.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:29.781 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:03:29.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:29.824 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-04-25T19:03:29.915 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:29.915 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:29.915 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T19:03:29.915 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T19:03:29.915 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:30.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:30.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:30.086 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T19:03:30.086 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T19:03:30.087 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:30.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:30.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:30.355 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:30.355 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:30.355 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-04-25T19:03:30.356 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T19:03:30.356 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:30.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:30.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:30.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:30.540 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T19:03:30.583 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T19:03:30.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:30.583 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T19:03:30.677 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T19:03:30.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:30.678 DEBUG:teuthology.orchestra.run.smithi146:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:30.900 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:30.900 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T19:03:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: trying mount with new device syntax: 1@a2d72801-e932-4284-9776-fce727cd5af1.cephfs=/ 2024-04-25T19:03:31.637 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.123:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T19:03:31.640 INFO:teuthology.orchestra.run.smithi146.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T19:03:31.640 INFO:teuthology.orchestra.run.smithi146.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:03:31.640 INFO:teuthology.orchestra.run.smithi146.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:03:31.641 INFO:teuthology.orchestra.run.smithi146.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:03:31.641 INFO:teuthology.orchestra.run.smithi146.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:03:31.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:03:31.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:31.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T19:03:31.730 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:31.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:31.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:31.852 INFO:teuthology.orchestra.run.smithi146.stdout:client4490 2024-04-25T19:03:31.852 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:03:31.852 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:31.903 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:32.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:32.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:32.019 INFO:teuthology.orchestra.run.smithi146.stdout:client4490 2024-04-25T19:03:32.019 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:32.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:32.130 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:32.139 INFO:teuthology.orchestra.run.smithi146.stdout:client4490 2024-04-25T19:03:32.140 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a2d72801-e932-4284-9776-fce727cd5af1.client4490/status') 2024-04-25T19:03:32.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:32.269 INFO:teuthology.orchestra.run.smithi146.stdout:instance: client.4490 (3)192.168.0.1:0/196900720 2024-04-25T19:03:32.269 INFO:teuthology.orchestra.run.smithi146.stdout:blocklisted: false 2024-04-25T19:03:32.269 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records in 2024-04-25T19:03:32.269 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records out 2024-04-25T19:03:32.269 INFO:teuthology.orchestra.run.smithi146.stderr:68 bytes copied, 7.5756e-05 s, 898 kB/s 2024-04-25T19:03:32.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:32.281 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T19:03:32.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:03:32.281 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T19:03:32.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:03:32.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.560+0000 7f108f200640 1 Processor -- start 2024-04-25T19:03:32.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.560+0000 7f108f200640 1 -- start start 2024-04-25T19:03:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.562+0000 7f108f200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f10900710a0 0x7f109015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.562+0000 7f108f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.562+0000 7f108f200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109006d750 con 0x7f109006d340 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.562+0000 7f108f200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f10900b6ea0 con 0x7f10900710a0 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.562+0000 7f108f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f109006f4b0 con 0x7f10900b7150 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.563+0000 7f108d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.563+0000 7f108d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:41838/0 (socket says 172.21.15.123:41838) 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.563+0000 7f108d400640 1 -- 172.21.15.123:0/3353870052 learned_addr learned my addr 172.21.15.123:0/3353870052 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.563+0000 7f108de00640 1 --2- 172.21.15.123:0/3353870052 >> v2:172.21.15.146:3300/0 conn(0x7f10900710a0 0x7f109015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.563+0000 7f108e800640 1 --1- 172.21.15.123:0/3353870052 >> v1:172.21.15.146:6789/0 conn(0x7f109006d340 0x7f109015fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:46570/0 (socket says 172.21.15.123:46570) 2024-04-25T19:03:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.564+0000 7f108d400640 1 -- 172.21.15.123:0/3353870052 >> v2:172.21.15.146:3300/0 conn(0x7f10900710a0 msgr2=0x7f109015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.564+0000 7f108d400640 1 --2- 172.21.15.123:0/3353870052 >> v2:172.21.15.146:3300/0 conn(0x7f10900710a0 0x7f109015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.564+0000 7f108d400640 1 -- 172.21.15.123:0/3353870052 >> v1:172.21.15.146:6789/0 conn(0x7f109006d340 legacy=0x7f109015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.564+0000 7f108d400640 1 -- 172.21.15.123:0/3353870052 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109006f790 con 0x7f10900b7150 2024-04-25T19:03:32.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.566+0000 7f108d400640 1 --2- 172.21.15.123:0/3353870052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f107c009e50 tx=0x7f107c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9908acd6acf663ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.566+0000 7f108ca00640 1 -- 172.21.15.123:0/3353870052 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c036070 con 0x7f10900b7150 2024-04-25T19:03:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.566+0000 7f108ca00640 1 -- 172.21.15.123:0/3353870052 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c031600 con 0x7f10900b7150 2024-04-25T19:03:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.566+0000 7f108ca00640 1 -- 172.21.15.123:0/3353870052 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c0319d0 con 0x7f10900b7150 2024-04-25T19:03:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 -- 172.21.15.123:0/3353870052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 msgr2=0x7f109006ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 --2- 172.21.15.123:0/3353870052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f107c009e50 tx=0x7f107c02a680 comp rx=0 tx=0).stop 2024-04-25T19:03:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 -- 172.21.15.123:0/3353870052 shutdown_connections 2024-04-25T19:03:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 --2- 172.21.15.123:0/3353870052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10900b7150 0x7f109006ce00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 --2- 172.21.15.123:0/3353870052 >> v2:172.21.15.146:3300/0 conn(0x7f10900710a0 0x7f109015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.567+0000 7f108f200640 1 -- 172.21.15.123:0/3353870052 >> 172.21.15.123:0/3353870052 conn(0x7f10900ce090 msgr2=0x7f10900ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:32.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.568+0000 7f108f200640 1 -- 172.21.15.123:0/3353870052 shutdown_connections 2024-04-25T19:03:32.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.568+0000 7f108f200640 1 -- 172.21.15.123:0/3353870052 wait complete. 2024-04-25T19:03:32.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.571+0000 7f108f200640 1 Processor -- start 2024-04-25T19:03:32.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.571+0000 7f108f200640 1 -- start start 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109006d340 0x7f10900c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10900b7150 0x7f10900cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1090070030 con 0x7f10900710a0 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f109006fd50 con 0x7f10900b7150 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f10900cf1e0 con 0x7f109006d340 2024-04-25T19:03:32.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42896/0 (socket says 172.21.15.123:42896) 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108d400640 1 -- 172.21.15.123:0/3571820085 learned_addr learned my addr 172.21.15.123:0/3571820085 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108de00640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109006d340 0x7f10900c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.573+0000 7f108e800640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10900b7150 0x7f10900cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 -- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109006d340 msgr2=0x7f10900c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109006d340 0x7f10900c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 -- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10900b7150 msgr2=0x7f10900cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10900b7150 0x7f10900cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 -- 172.21.15.123:0/3571820085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10900c6e50 con 0x7f10900710a0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108d400640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f107c009f70 tx=0x7f107c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c041020 con 0x7f10900710a0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c007820 con 0x7f10900710a0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10901fa110 con 0x7f10900710a0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.574+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c036040 con 0x7f10900710a0 2024-04-25T19:03:32.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.575+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10901fa6d0 con 0x7f10900710a0 2024-04-25T19:03:32.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.576+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1090070720 con 0x7f10900710a0 2024-04-25T19:03:32.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.577+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f107c044020 con 0x7f10900710a0 2024-04-25T19:03:32.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.578+0000 7f1086000640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 0x7f10700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:32.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.578+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f107c0b6310 con 0x7f10900710a0 2024-04-25T19:03:32.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.578+0000 7f108de00640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 0x7f10700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:32.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.579+0000 7f108de00640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 0x7f10700769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10900cec10 tx=0x7f1080008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.583+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f107c04b030 con 0x7f10900710a0 2024-04-25T19:03:32.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.830+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f10900b0b90 con 0x7f10900710a0 2024-04-25T19:03:32.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.831+0000 7f1086000640 1 -- 172.21.15.123:0/3571820085 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f107c080610 con 0x7f10900710a0 2024-04-25T19:03:32.833 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T19:03:32.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 msgr2=0x7f10700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 0x7f10700769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10900cec10 tx=0x7f1080008040 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 msgr2=0x7f1090148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f107c009f70 tx=0x7f107c004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 shutdown_connections 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f10700744b0 0x7f10700769c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10900b7150 0x7f10900cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10900710a0 0x7f1090148010 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 --2- 172.21.15.123:0/3571820085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109006d340 0x7f10900c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.834+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 >> 172.21.15.123:0/3571820085 conn(0x7f10900ce090 msgr2=0x7f10900b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.835+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 shutdown_connections 2024-04-25T19:03:32.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:32.835+0000 7f108f200640 1 -- 172.21.15.123:0/3571820085 wait complete. 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:03:32.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:03:32.863 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T19:03:32.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:32.863 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-04-25T19:03:32.919 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:32.919 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:32.920 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42605sec preferred_lft 42605sec 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 3e:36:3d:bb:43:05 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a030:e6ff:fe7b:1fac/64 scope link 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.921 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T19:03:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 3e:36:3d:bb:43:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T19:03:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::3c36:3dff:febb:4305/64 scope link 2024-04-25T19:03:32.922 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:32.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:32.923 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:03:32.996 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T19:03:33.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:33.000 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-04-25T19:03:33.052 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-04-25T19:03:33.052 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:03:33.052 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:03:33.052 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:03:33.053 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:03:33.053 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:33.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:33.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:33.260 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:03:33.306 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T19:03:33.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:33.308 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-04-25T19:03:33.352 INFO:teuthology.orchestra.run.smithi146.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T19:03:33.353 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:33.353 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:33.353 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T19:03:33.353 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T19:03:33.353 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:33.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:33.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:33.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:33.582 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 9e:35:bd:a2:26:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.656 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::9c35:bdff:fea2:266e/64 scope link 2024-04-25T19:03:33.657 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:33.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T19:03:33.743 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:03:33.743 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:03:33.743 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 9e:35:bd:a2:26:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::9c35:bdff:fea2:266e/64 scope link 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:03:33.748 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T19:03:33.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T19:03:33.749 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:33.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:34.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:34.022 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:03:34.022 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T19:03:34.023 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.1 up 2024-04-25T19:03:34.023 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T19:03:34.023 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T19:03:34.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:34.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:34.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:34.225 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T19:03:34.272 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T19:03:34.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:03:34.272 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T19:03:34.364 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T19:03:34.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:34.365 DEBUG:teuthology.orchestra.run.smithi146:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:34.566 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:34.566 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:34.608 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:03:34.608 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:03:34.608 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T19:03:34.608 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: trying mount with new device syntax: 3@a2d72801-e932-4284-9776-fce727cd5af1.cephfs=/ 2024-04-25T19:03:34.608 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.123:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T19:03:34.609 INFO:teuthology.orchestra.run.smithi146.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T19:03:34.610 INFO:teuthology.orchestra.run.smithi146.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:03:34.610 INFO:teuthology.orchestra.run.smithi146.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:03:34.610 INFO:teuthology.orchestra.run.smithi146.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:03:34.610 INFO:teuthology.orchestra.run.smithi146.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:03:34.614 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:03:34.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:34.615 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T19:03:34.690 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:34.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:34.801 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:34.810 INFO:teuthology.orchestra.run.smithi146.stdout:client4839 2024-04-25T19:03:34.811 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:03:34.811 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:34.862 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:34.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:34.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:34.980 INFO:teuthology.orchestra.run.smithi146.stdout:client4839 2024-04-25T19:03:34.980 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:03:35.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:03:35.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:35.105 INFO:teuthology.orchestra.run.smithi146.stdout:client4839 2024-04-25T19:03:35.105 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a2d72801-e932-4284-9776-fce727cd5af1.client4839/status') 2024-04-25T19:03:35.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:03:35.230 INFO:teuthology.orchestra.run.smithi146.stdout:instance: client.4839 (3)172.21.15.146:0/2850721123 2024-04-25T19:03:35.230 INFO:teuthology.orchestra.run.smithi146.stdout:blocklisted: false 2024-04-25T19:03:35.231 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records in 2024-04-25T19:03:35.231 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records out 2024-04-25T19:03:35.231 INFO:teuthology.orchestra.run.smithi146.stderr:71 bytes copied, 8.0182e-05 s, 885 kB/s 2024-04-25T19:03:35.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T19:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:03:35.243 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T19:03:35.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:35.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.544+0000 7f4486200640 1 Processor -- start 2024-04-25T19:03:35.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.545+0000 7f4486200640 1 -- start start 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.547+0000 7f4486200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4480147880 0x7f4480147c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.547+0000 7f4486200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4480164c40 0x7f448006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.547+0000 7f4486200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4480165360 con 0x7f4480164830 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.547+0000 7f4486200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4480165640 con 0x7f4480164c40 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.547+0000 7f4486200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f44800d02a0 con 0x7f4480147880 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4480164c40 0x7f448006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4480164c40 0x7f448006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51990/0 (socket says 172.21.15.123:51990) 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 -- 172.21.15.123:0/325810195 learned_addr learned my addr 172.21.15.123:0/325810195 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4484e00640 1 --2- 172.21.15.123:0/325810195 >> v2:172.21.15.146:3300/0 conn(0x7f4480147880 0x7f4480147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:35.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 -- 172.21.15.123:0/325810195 >> v2:172.21.15.146:3300/0 conn(0x7f4480147880 msgr2=0x7f4480147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:35.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 --2- 172.21.15.123:0/325810195 >> v2:172.21.15.146:3300/0 conn(0x7f4480147880 0x7f4480147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 -- 172.21.15.123:0/325810195 >> v1:172.21.15.146:6789/0 conn(0x7f4480164830 legacy=0x7f44800d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:35.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.548+0000 7f4485800640 1 -- 172.21.15.123:0/325810195 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44800d0580 con 0x7f4480164c40 2024-04-25T19:03:35.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.549+0000 7f4485800640 1 --2- 172.21.15.123:0/325810195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448006d580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4480074310 tx=0x7f447802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17969302dec161b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:35.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.550+0000 7f447fe00640 1 -- 172.21.15.123:0/325810195 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478036070 con 0x7f4480164c40 2024-04-25T19:03:35.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.550+0000 7f447fe00640 1 -- 172.21.15.123:0/325810195 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4478031600 con 0x7f4480164c40 2024-04-25T19:03:35.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.550+0000 7f447fe00640 1 -- 172.21.15.123:0/325810195 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44780319d0 con 0x7f4480164c40 2024-04-25T19:03:35.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.550+0000 7f4486200640 1 -- 172.21.15.123:0/325810195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 msgr2=0x7f448006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:35.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.550+0000 7f4486200640 1 --2- 172.21.15.123:0/325810195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448006d580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4480074310 tx=0x7f447802a660 comp rx=0 tx=0).stop 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 -- 172.21.15.123:0/325810195 shutdown_connections 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 --2- 172.21.15.123:0/325810195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448006d580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 --2- 172.21.15.123:0/325810195 >> v2:172.21.15.146:3300/0 conn(0x7f4480147880 0x7f4480147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 -- 172.21.15.123:0/325810195 >> 172.21.15.123:0/325810195 conn(0x7f4480073200 msgr2=0x7f4480073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 -- 172.21.15.123:0/325810195 shutdown_connections 2024-04-25T19:03:35.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.551+0000 7f4486200640 1 -- 172.21.15.123:0/325810195 wait complete. 2024-04-25T19:03:35.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.553+0000 7f4486200640 1 Processor -- start 2024-04-25T19:03:35.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.553+0000 7f4486200640 1 -- start start 2024-04-25T19:03:35.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4480147880 0x7f448015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 0x7f448015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44800ce390 con 0x7f4480164c40 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4480165050 con 0x7f4480164830 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4486200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f44800ce510 con 0x7f4480147880 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51996/0 (socket says 172.21.15.123:51996) 2024-04-25T19:03:35.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 -- 172.21.15.123:0/1265673165 learned_addr learned my addr 172.21.15.123:0/1265673165 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:35.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4477e00640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 0x7f448015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:35.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 -- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4480147880 msgr2=0x7f448015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:35.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4480147880 0x7f448015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 -- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 msgr2=0x7f448015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 0x7f448015ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4485800640 1 -- 172.21.15.123:0/1265673165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4480161c00 con 0x7f4480164c40 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.555+0000 7f4477e00640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 0x7f448015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f4485800640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4478034040 tx=0x7f4478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478041020 con 0x7f4480164c40 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4478007780 con 0x7f4480164c40 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478036040 con 0x7f4480164c40 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4480161ee0 con 0x7f4480164c40 2024-04-25T19:03:35.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.556+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44801624a0 con 0x7f4480164c40 2024-04-25T19:03:35.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.557+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4480050740 con 0x7f4480164c40 2024-04-25T19:03:35.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.559+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4478044020 con 0x7f4480164c40 2024-04-25T19:03:35.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.560+0000 7f447d600640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 0x7f445c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:35.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.560+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f44780b64e0 con 0x7f4480164c40 2024-04-25T19:03:35.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.560+0000 7f4484e00640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 0x7f445c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:35.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.561+0000 7f4484e00640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 0x7f445c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4480070430 tx=0x7f4470008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:35.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.565+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f447804b030 con 0x7f4480164c40 2024-04-25T19:03:35.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.767+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4480073990 con 0x7f4480164c40 2024-04-25T19:03:35.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.770+0000 7f447d600640 1 -- 172.21.15.123:0/1265673165 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f4478080920 con 0x7f4480164c40 2024-04-25T19:03:35.772 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:35.772 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 msgr2=0x7f445c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 0x7f445c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4480070430 tx=0x7f4470008040 comp rx=0 tx=0).stop 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 msgr2=0x7f448015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4478034040 tx=0x7f4478004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 shutdown_connections 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f445c0744b0 0x7f445c0769c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480164c40 0x7f448015f9d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4480164830 0x7f448015ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 --2- 172.21.15.123:0/1265673165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4480147880 0x7f448015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:35.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 >> 172.21.15.123:0/1265673165 conn(0x7f4480073200 msgr2=0x7f44801664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:35.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 shutdown_connections 2024-04-25T19:03:35.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:35.773+0000 7f4486200640 1 -- 172.21.15.123:0/1265673165 wait complete. 2024-04-25T19:03:35.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:35.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.082+0000 7f1c73e00640 1 Processor -- start 2024-04-25T19:03:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.082+0000 7f1c73e00640 1 -- start start 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1c740b0700 0x7f1c740702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c7406d580 0x7f1c7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c740d1430 con 0x7f1c7415df10 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1c74070f40 con 0x7f1c7406d580 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1c74161e90 con 0x7f1c740b0700 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c72a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1c7415df10 0x7f1c74162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49756/0 (socket says 172.21.15.123:49756) 2024-04-25T19:03:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c72a00640 1 -- 172.21.15.123:0/825583590 learned_addr learned my addr 172.21.15.123:0/825583590 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c73400640 1 --2- 172.21.15.123:0/825583590 >> v2:172.21.15.123:3300/0 conn(0x7f1c7406d580 0x7f1c7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.085+0000 7f1c72000640 1 --2- 172.21.15.123:0/825583590 >> v2:172.21.15.146:3300/0 conn(0x7f1c740b0700 0x7f1c740702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.086+0000 7f1c73400640 1 -- 172.21.15.123:0/825583590 >> v2:172.21.15.146:3300/0 conn(0x7f1c740b0700 msgr2=0x7f1c740702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.086+0000 7f1c73400640 1 --2- 172.21.15.123:0/825583590 >> v2:172.21.15.146:3300/0 conn(0x7f1c740b0700 0x7f1c740702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.086+0000 7f1c73400640 1 -- 172.21.15.123:0/825583590 >> v1:172.21.15.123:6789/0 conn(0x7f1c7415df10 legacy=0x7f1c74162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.086+0000 7f1c73400640 1 -- 172.21.15.123:0/825583590 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7406ded0 con 0x7f1c7406d580 2024-04-25T19:03:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.087+0000 7f1c73400640 1 --2- 172.21.15.123:0/825583590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c740694a0 tx=0x7f1c6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f979df47fa8034d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.087+0000 7f1c71600640 1 -- 172.21.15.123:0/825583590 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c037070 con 0x7f1c7406d580 2024-04-25T19:03:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.087+0000 7f1c71600640 1 -- 172.21.15.123:0/825583590 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c032440 con 0x7f1c7406d580 2024-04-25T19:03:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.087+0000 7f1c71600640 1 -- 172.21.15.123:0/825583590 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c032810 con 0x7f1c7406d580 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 -- 172.21.15.123:0/825583590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 msgr2=0x7f1c7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 --2- 172.21.15.123:0/825583590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c740694a0 tx=0x7f1c6c02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 -- 172.21.15.123:0/825583590 shutdown_connections 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 --2- 172.21.15.123:0/825583590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7406d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 --2- 172.21.15.123:0/825583590 >> v2:172.21.15.146:3300/0 conn(0x7f1c740b0700 0x7f1c740702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 -- 172.21.15.123:0/825583590 >> 172.21.15.123:0/825583590 conn(0x7f1c74073090 msgr2=0x7f1c740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 -- 172.21.15.123:0/825583590 shutdown_connections 2024-04-25T19:03:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.088+0000 7f1c73e00640 1 -- 172.21.15.123:0/825583590 wait complete. 2024-04-25T19:03:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.091+0000 7f1c73e00640 1 Processor -- start 2024-04-25T19:03:36.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.091+0000 7f1c73e00640 1 -- start start 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c740b0700 0x7f1c740ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 0x7f1c7415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74161b20 con 0x7f1c7406d580 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74070c60 con 0x7f1c7415df10 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.092+0000 7f1c73e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c740cdfe0 con 0x7f1c740b0700 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52012/0 (socket says 172.21.15.123:52012) 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 -- 172.21.15.123:0/4219438140 learned_addr learned my addr 172.21.15.123:0/4219438140 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c73400640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 0x7f1c7415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72000640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c740b0700 0x7f1c740ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 -- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c740b0700 msgr2=0x7f1c740ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c740b0700 0x7f1c740ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 -- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 msgr2=0x7f1c7415c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 0x7f1c7415c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 -- 172.21.15.123:0/4219438140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c741f42b0 con 0x7f1c7406d580 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c73400640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 0x7f1c7415c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:36.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.093+0000 7f1c72a00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c440 tx=0x7f1c6400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.094+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64017070 con 0x7f1c7406d580 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.094+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c64005ce0 con 0x7f1c7406d580 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.094+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640072f0 con 0x7f1c7406d580 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.094+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741f4570 con 0x7f1c7406d580 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.094+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741f4b30 con 0x7f1c7406d580 2024-04-25T19:03:36.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.095+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c74050740 con 0x7f1c7406d580 2024-04-25T19:03:36.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.096+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c6401a070 con 0x7f1c7406d580 2024-04-25T19:03:36.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.097+0000 7f1c6aa00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 0x7f1c500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.097+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1c64094970 con 0x7f1c7406d580 2024-04-25T19:03:36.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.097+0000 7f1c72000640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 0x7f1c500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.098+0000 7f1c72000640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 0x7f1c500769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c740b3130 tx=0x7f1c5c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.102+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c6405ecf0 con 0x7f1c7406d580 2024-04-25T19:03:36.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.301+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c74074190 con 0x7f1c7406d580 2024-04-25T19:03:36.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.302+0000 7f1c6aa00640 1 -- 172.21.15.123:0/4219438140 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f1c6405e5f0 con 0x7f1c7406d580 2024-04-25T19:03:36.304 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 msgr2=0x7f1c500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 0x7f1c500769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c740b3130 tx=0x7f1c5c00b040 comp rx=0 tx=0).stop 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 msgr2=0x7f1c7414ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c440 tx=0x7f1c6400c900 comp rx=0 tx=0).stop 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 shutdown_connections 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c500744e0 0x7f1c500769f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c7415df10 0x7f1c7415c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c740b0700 0x7f1c740ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 --2- 172.21.15.123:0/4219438140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7406d580 0x7f1c7414ef90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 >> 172.21.15.123:0/4219438140 conn(0x7f1c74073090 msgr2=0x7f1c7406ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 shutdown_connections 2024-04-25T19:03:36.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.305+0000 7f1c73e00640 1 -- 172.21.15.123:0/4219438140 wait complete. 2024-04-25T19:03:36.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:36.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.604+0000 7f48d1200640 1 Processor -- start 2024-04-25T19:03:36.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.605+0000 7f48d1200640 1 -- start start 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48d1200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f48cc15df10 0x7f48cc0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48d1200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48cc162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48d1200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0d1430 con 0x7f48cc06d580 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48d1200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc070f40 con 0x7f48cc15df10 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48d1200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc161e90 con 0x7f48cc0b0700 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48cbe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f48cc06d580 0x7f48cc06d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49772/0 (socket says 172.21.15.123:49772) 2024-04-25T19:03:36.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.607+0000 7f48cbe00640 1 -- 172.21.15.123:0/3260108459 learned_addr learned my addr 172.21.15.123:0/3260108459 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:36.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48cb400640 1 --2- 172.21.15.123:0/3260108459 >> v2:172.21.15.146:3300/0 conn(0x7f48cc15df10 0x7f48cc0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48c2a00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48cc162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678364769 0 0) 0x7f48cc0d1430 con 0x7f48cc06d580 2024-04-25T19:03:36.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48b0003520 con 0x7f48cc06d580 2024-04-25T19:03:36.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1980323276 0 0) 0x7f48b0003520 con 0x7f48cc06d580 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 msgr2=0x7f48cc162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48cc162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 >> v2:172.21.15.146:3300/0 conn(0x7f48cc15df10 msgr2=0x7f48cc0702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 --2- 172.21.15.123:0/3260108459 >> v2:172.21.15.146:3300/0 conn(0x7f48cc15df10 0x7f48cc0702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc070980 con 0x7f48cc06d580 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f48c40019f0 con 0x7f48cc06d580 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 >> v1:172.21.15.123:6789/0 conn(0x7f48cc06d580 legacy=0x7f48cc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0003f30 0x7f48b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48b0004d40 0x7f48b0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc161e90 con 0x7f48b0004d40 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48b000ba70 con 0x7f48b0003f30 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.608+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f48b000bc40 con 0x7f48cc0b0700 2024-04-25T19:03:36.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48b0004d40 0x7f48b0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cb400640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48c2a00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0003f30 0x7f48b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 -- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 msgr2=0x7f48b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0b0700 0x7f48b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 -- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0003f30 msgr2=0x7f48b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0003f30 0x7f48b0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.609+0000 7f48cbe00640 1 -- 172.21.15.123:0/3260108459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b0003520 con 0x7f48b0004d40 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.610+0000 7f48cbe00640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48b0004d40 0x7f48b0005150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48c400a8d0 tx=0x7f48b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2838cb32b06640a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.610+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400a4c0 con 0x7f48b0004d40 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.610+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c402f070 con 0x7f48b0004d40 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.610+0000 7f48caa00640 1 -- 172.21.15.123:0/3260108459 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c402bd40 con 0x7f48b0004d40 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 -- 172.21.15.123:0/3260108459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48b0004d40 msgr2=0x7f48b0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 --2- 172.21.15.123:0/3260108459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48b0004d40 0x7f48b0005150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48c400a8d0 tx=0x7f48b0005830 comp rx=0 tx=0).stop 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48cbe00640 1 -- 172.21.15.123:0/3260108459 reap_dead start 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 -- 172.21.15.123:0/3260108459 shutdown_connections 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 -- 172.21.15.123:0/3260108459 >> 172.21.15.123:0/3260108459 conn(0x7f48cc073090 msgr2=0x7f48cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 -- 172.21.15.123:0/3260108459 shutdown_connections 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.611+0000 7f48d1200640 1 -- 172.21.15.123:0/3260108459 wait complete. 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.614+0000 7f48d1200640 1 Processor -- start 2024-04-25T19:03:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.614+0000 7f48d1200640 1 -- start start 2024-04-25T19:03:36.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48b0004d40 0x7f48cc1e9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15be90 0x7f48cc15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc162320 con 0x7f48cc1e75e0 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc161b20 con 0x7f48cc15be90 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.615+0000 7f48d1200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc1624a0 con 0x7f48b0004d40 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52040/0 (socket says 172.21.15.123:52040) 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 -- 172.21.15.123:0/519239488 learned_addr learned my addr 172.21.15.123:0/519239488 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48cbe00640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15be90 0x7f48cc15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 -- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48b0004d40 msgr2=0x7f48cc1e9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48b0004d40 0x7f48cc1e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 -- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15be90 msgr2=0x7f48cc15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15be90 0x7f48cc15c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 -- 172.21.15.123:0/519239488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc15c7e0 con 0x7f48cc1e75e0 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.616+0000 7f48c2a00640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48b400e810 tx=0x7f48b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.617+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4017070 con 0x7f48cc1e75e0 2024-04-25T19:03:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.617+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b400c6d0 con 0x7f48cc1e75e0 2024-04-25T19:03:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.617+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b400f040 con 0x7f48cc1e75e0 2024-04-25T19:03:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.617+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc1e8060 con 0x7f48cc1e75e0 2024-04-25T19:03:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.617+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1e8620 con 0x7f48cc1e75e0 2024-04-25T19:03:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.618+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc050740 con 0x7f48cc1e75e0 2024-04-25T19:03:36.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.619+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48b401a070 con 0x7f48cc1e75e0 2024-04-25T19:03:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.620+0000 7f48c3e00640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 0x7f48a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.620+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f48b4095190 con 0x7f48cc1e75e0 2024-04-25T19:03:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.620+0000 7f48cb400640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 0x7f48a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.621+0000 7f48cb400640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 0x7f48a80769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48cc1618e0 tx=0x7f48bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.625+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48cc15c7e0 con 0x7f48cc1e75e0 2024-04-25T19:03:36.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.824+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48cc15d370 con 0x7f48cc1e75e0 2024-04-25T19:03:36.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.825+0000 7f48c3e00640 1 -- 172.21.15.123:0/519239488 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f48b405f5a0 con 0x7f48cc1e75e0 2024-04-25T19:03:36.827 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.827+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 msgr2=0x7f48a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.827+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 0x7f48a80769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48cc1618e0 tx=0x7f48bc008040 comp rx=0 tx=0).stop 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.827+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 msgr2=0x7f48cc1e79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.827+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48b400e810 tx=0x7f48b400ecd0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 shutdown_connections 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f48a80744e0 0x7f48a80769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15be90 0x7f48cc15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48cc1e75e0 0x7f48cc1e79f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 --2- 172.21.15.123:0/519239488 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48b0004d40 0x7f48cc1e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 >> 172.21.15.123:0/519239488 conn(0x7f48cc073090 msgr2=0x7f48cc0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 shutdown_connections 2024-04-25T19:03:36.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:36.828+0000 7f48d1200640 1 -- 172.21.15.123:0/519239488 wait complete. 2024-04-25T19:03:36.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:37.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.119+0000 7f253be00640 1 Processor -- start 2024-04-25T19:03:37.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.119+0000 7f253be00640 1 -- start start 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.121+0000 7f253be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f253c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.121+0000 7f253be00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253c070a20 con 0x7f253c06fcd0 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.121+0000 7f253be00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253c06c780 con 0x7f253c0cf500 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.121+0000 7f253be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f253c0707f0 con 0x7f253c0b08c0 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f253c06fcd0 0x7f253c0700e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49778/0 (socket says 172.21.15.123:49778) 2024-04-25T19:03:37.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253b400640 1 -- 172.21.15.123:0/1513045304 learned_addr learned my addr 172.21.15.123:0/1513045304 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253aa00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f253c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741650226 0 0) 0x7f253c070a20 con 0x7f253c06fcd0 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2520003520 con 0x7f253c06fcd0 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.122+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3833233303 0 0) 0x7f2520003520 con 0x7f253c06fcd0 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 msgr2=0x7f253c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f253c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 >> v1:172.21.15.146:6789/0 conn(0x7f253c0cf500 legacy=0x7f253c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c06ca60 con 0x7f253c06fcd0 2024-04-25T19:03:37.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f25340019f0 con 0x7f253c06fcd0 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 >> v1:172.21.15.123:6789/0 conn(0x7f253c06fcd0 legacy=0x7f253c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f25200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2520003f60 0x7f2520004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520004d40 0x7f2520005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f252000bb50 con 0x7f2520003f60 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f252000bcd0 con 0x7f2520004d40 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f252000bed0 con 0x7f253c0b08c0 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2520003f60 0x7f2520004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253aa00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f25200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 -- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 msgr2=0x7f25200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f25200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 -- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520004d40 msgr2=0x7f2520005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520004d40 0x7f2520005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f2533e00640 1 -- 172.21.15.123:0/1513045304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2520003520 con 0x7f2520003f60 2024-04-25T19:03:37.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.123+0000 7f253aa00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c0b08c0 0x7f25200039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:37.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.124+0000 7f2533e00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2520003f60 0x7f2520004800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2524001750 tx=0x7f2520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13decdf8c1d1a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524001370 con 0x7f2520003f60 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25240052d0 con 0x7f2520003f60 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253a000640 1 -- 172.21.15.123:0/1513045304 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252402abe0 con 0x7f2520003f60 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253be00640 1 -- 172.21.15.123:0/1513045304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2520003f60 msgr2=0x7f2520004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253be00640 1 --2- 172.21.15.123:0/1513045304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2520003f60 0x7f2520004800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2524001750 tx=0x7f2520005830 comp rx=0 tx=0).stop 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253b400640 1 -- 172.21.15.123:0/1513045304 reap_dead start 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253be00640 1 -- 172.21.15.123:0/1513045304 shutdown_connections 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.125+0000 7f253be00640 1 -- 172.21.15.123:0/1513045304 >> 172.21.15.123:0/1513045304 conn(0x7f253c0d19f0 msgr2=0x7f253c0d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.126+0000 7f253be00640 1 -- 172.21.15.123:0/1513045304 shutdown_connections 2024-04-25T19:03:37.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.126+0000 7f253be00640 1 -- 172.21.15.123:0/1513045304 wait complete. 2024-04-25T19:03:37.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.128+0000 7f253be00640 1 Processor -- start 2024-04-25T19:03:37.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.128+0000 7f253be00640 1 -- start start 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 0x7f253c14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c14cf50 0x7f253c145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f253c0cd9b0 con 0x7f253c1519e0 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f253c0cd830 con 0x7f2520003f60 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f253c0cdb30 con 0x7f253c14cf50 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52066/0 (socket says 172.21.15.123:52066) 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 -- 172.21.15.123:0/4012677940 learned_addr learned my addr 172.21.15.123:0/4012677940 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253aa00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 0x7f253c14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f2533e00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c14cf50 0x7f253c145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 -- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c14cf50 msgr2=0x7f253c145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c14cf50 0x7f253c145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 -- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 msgr2=0x7f253c14ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 0x7f253c14ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253b400640 1 -- 172.21.15.123:0/4012677940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c150940 con 0x7f253c1519e0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.130+0000 7f253aa00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 0x7f253c14ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f253b400640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2534001370 tx=0x7f253400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253400a5b0 con 0x7f253c1519e0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253400a750 con 0x7f253c1519e0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253400ab00 con 0x7f253c1519e0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c150c20 con 0x7f253c1519e0 2024-04-25T19:03:37.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.131+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c1511e0 con 0x7f253c1519e0 2024-04-25T19:03:37.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.132+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253c050740 con 0x7f253c1519e0 2024-04-25T19:03:37.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.134+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f253400aca0 con 0x7f253c1519e0 2024-04-25T19:03:37.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.134+0000 7f2533400640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.135+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f25340947c0 con 0x7f253c1519e0 2024-04-25T19:03:37.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.135+0000 7f253aa00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.135+0000 7f253aa00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 0x7f25180769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f253c071530 tx=0x7f252c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.139+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f253401a070 con 0x7f253c1519e0 2024-04-25T19:03:37.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.334+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f253c158750 con 0x7f253c1519e0 2024-04-25T19:03:37.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.335+0000 7f2533400640 1 -- 172.21.15.123:0/4012677940 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f253405ec70 con 0x7f253c1519e0 2024-04-25T19:03:37.337 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 msgr2=0x7f25180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 0x7f25180769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f253c071530 tx=0x7f252c008040 comp rx=0 tx=0).stop 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 msgr2=0x7f253c146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2534001370 tx=0x7f253400ccb0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 shutdown_connections 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f253c1519e0 0x7f253c146380 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f253c14cf50 0x7f253c145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 --2- 172.21.15.123:0/4012677940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2520003f60 0x7f253c14ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 >> 172.21.15.123:0/4012677940 conn(0x7f253c0d19f0 msgr2=0x7f253c14de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 shutdown_connections 2024-04-25T19:03:37.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.338+0000 7f253be00640 1 -- 172.21.15.123:0/4012677940 wait complete. 2024-04-25T19:03:37.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:37.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.636+0000 7faaef000640 1 Processor -- start 2024-04-25T19:03:37.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.636+0000 7faaef000640 1 -- start start 2024-04-25T19:03:37.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.638+0000 7faaef000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faae815df10 0x7faae80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.638+0000 7faaef000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faae80b0700 0x7faae8162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.638+0000 7faaef000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae80d1430 con 0x7faae806d580 2024-04-25T19:03:37.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.638+0000 7faaef000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faae8070f40 con 0x7faae80b0700 2024-04-25T19:03:37.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.638+0000 7faaef000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faae8161e90 con 0x7faae815df10 2024-04-25T19:03:37.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.639+0000 7faaed200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faae80b0700 0x7faae8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.639+0000 7faaed200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faae80b0700 0x7faae8162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52076/0 (socket says 172.21.15.123:52076) 2024-04-25T19:03:37.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.639+0000 7faaed200640 1 -- 172.21.15.123:0/1167015191 learned_addr learned my addr 172.21.15.123:0/1167015191 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:37.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.639+0000 7faaedc00640 1 --2- 172.21.15.123:0/1167015191 >> v2:172.21.15.146:3300/0 conn(0x7faae815df10 0x7faae80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.640+0000 7faaed200640 1 -- 172.21.15.123:0/1167015191 >> v2:172.21.15.146:3300/0 conn(0x7faae815df10 msgr2=0x7faae80702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.640+0000 7faaed200640 1 --2- 172.21.15.123:0/1167015191 >> v2:172.21.15.146:3300/0 conn(0x7faae815df10 0x7faae80702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.640+0000 7faaed200640 1 -- 172.21.15.123:0/1167015191 >> v1:172.21.15.146:6789/0 conn(0x7faae806d580 legacy=0x7faae806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.640+0000 7faaed200640 1 -- 172.21.15.123:0/1167015191 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae8070980 con 0x7faae80b0700 2024-04-25T19:03:37.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.641+0000 7faaed200640 1 --2- 172.21.15.123:0/1167015191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80b0700 0x7faae8162320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faae80517e0 tx=0x7faad402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6ed9da318689f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.641+0000 7faae7e00640 1 -- 172.21.15.123:0/1167015191 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4037070 con 0x7faae80b0700 2024-04-25T19:03:37.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.641+0000 7faae7e00640 1 -- 172.21.15.123:0/1167015191 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad4032440 con 0x7faae80b0700 2024-04-25T19:03:37.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.641+0000 7faae7e00640 1 -- 172.21.15.123:0/1167015191 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4032810 con 0x7faae80b0700 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 -- 172.21.15.123:0/1167015191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80b0700 msgr2=0x7faae8162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 --2- 172.21.15.123:0/1167015191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80b0700 0x7faae8162320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faae80517e0 tx=0x7faad402a270 comp rx=0 tx=0).stop 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 -- 172.21.15.123:0/1167015191 shutdown_connections 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 --2- 172.21.15.123:0/1167015191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae80b0700 0x7faae8162320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 --2- 172.21.15.123:0/1167015191 >> v2:172.21.15.146:3300/0 conn(0x7faae815df10 0x7faae80702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 -- 172.21.15.123:0/1167015191 >> 172.21.15.123:0/1167015191 conn(0x7faae8073090 msgr2=0x7faae80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 -- 172.21.15.123:0/1167015191 shutdown_connections 2024-04-25T19:03:37.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.642+0000 7faaef000640 1 -- 172.21.15.123:0/1167015191 wait complete. 2024-04-25T19:03:37.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.645+0000 7faaef000640 1 Processor -- start 2024-04-25T19:03:37.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.645+0000 7faaef000640 1 -- start start 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faae80b0700 0x7faae8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faae815df10 0x7faae81f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faae8162860 con 0x7faae806d580 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faae8161b20 con 0x7faae815df10 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.646+0000 7faaef000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faae80cdfe0 con 0x7faae80b0700 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52078/0 (socket says 172.21.15.123:52078) 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 -- 172.21.15.123:0/1291937638 learned_addr learned my addr 172.21.15.123:0/1291937638 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 -- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faae80b0700 msgr2=0x7faae8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faae80b0700 0x7faae8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 -- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faae815df10 msgr2=0x7faae81f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faae815df10 0x7faae81f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 -- 172.21.15.123:0/1291937638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81f79b0 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.647+0000 7faaedc00640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d5d0 tx=0x7faadc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.648+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc004240 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.648+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc0043e0 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.648+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc003830 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.648+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae81f7c90 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.648+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae81f8250 con 0x7faae806d580 2024-04-25T19:03:37.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.649+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8050740 con 0x7faae806d580 2024-04-25T19:03:37.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.650+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faadc0039d0 con 0x7faae806d580 2024-04-25T19:03:37.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.651+0000 7faae5600640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.652+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7faadc095820 con 0x7faae806d580 2024-04-25T19:03:37.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.652+0000 7faaed200640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.652+0000 7faaed200640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 0x7faac80769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.656+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faadc05fcd0 con 0x7faae806d580 2024-04-25T19:03:37.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.851+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faae8074190 con 0x7faae806d580 2024-04-25T19:03:37.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.852+0000 7faae5600640 1 -- 172.21.15.123:0/1291937638 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7faadc05f5d0 con 0x7faae806d580 2024-04-25T19:03:37.854 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 msgr2=0x7faac80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 0x7faac80769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402c000 comp rx=0 tx=0).stop 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 msgr2=0x7faae81f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d5d0 tx=0x7faadc00da90 comp rx=0 tx=0).stop 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 shutdown_connections 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7faac80744e0 0x7faac80769f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faae815df10 0x7faae81f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faae80b0700 0x7faae8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 --2- 172.21.15.123:0/1291937638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faae806d580 0x7faae81f6bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 >> 172.21.15.123:0/1291937638 conn(0x7faae8073090 msgr2=0x7faae80b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 shutdown_connections 2024-04-25T19:03:37.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:37.855+0000 7faaef000640 1 -- 172.21.15.123:0/1291937638 wait complete. 2024-04-25T19:03:37.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:38.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.133+0000 7fd887e00640 1 Processor -- start 2024-04-25T19:03:38.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.133+0000 7fd887e00640 1 -- start start 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd887e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8880b4b30 0x7fd8880d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd887e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd888160480 con 0x7fd888160070 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd887e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd88806c710 con 0x7fd8880d0750 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd887e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd88806c9f0 con 0x7fd8880b4b30 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd886000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd888160070 0x7fd8880b8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49792/0 (socket says 172.21.15.123:49792) 2024-04-25T19:03:38.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.135+0000 7fd886000640 1 -- 172.21.15.123:0/2692993273 learned_addr learned my addr 172.21.15.123:0/2692993273 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:38.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794388944 0 0) 0x7fd888160480 con 0x7fd888160070 2024-04-25T19:03:38.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd886a00640 1 --2- 172.21.15.123:0/2692993273 >> v2:172.21.15.146:3300/0 conn(0x7fd8880b4b30 0x7fd8880d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd86c003520 con 0x7fd888160070 2024-04-25T19:03:38.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125023914 0 0) 0x7fd86c003520 con 0x7fd888160070 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 >> v2:172.21.15.146:3300/0 conn(0x7fd8880b4b30 msgr2=0x7fd8880d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 --2- 172.21.15.123:0/2692993273 >> v2:172.21.15.146:3300/0 conn(0x7fd8880b4b30 0x7fd8880d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 >> v1:172.21.15.146:6789/0 conn(0x7fd8880d0750 legacy=0x7fd8880d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88806ccd0 con 0x7fd888160070 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fd874001670 con 0x7fd888160070 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 >> v1:172.21.15.123:6789/0 conn(0x7fd888160070 legacy=0x7fd8880b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.136+0000 7fd885600640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c0039b0 0x7fd86c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd885600640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 0x7fd86c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd885600640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd86c005300 0x7fd86c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd88806c9f0 con 0x7fd86c004800 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c00c2a0 con 0x7fd86c0039b0 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c00c4a0 con 0x7fd86c005300 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 0x7fd86c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 -- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd86c005300 msgr2=0x7fd86c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:38.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd86c005300 0x7fd86c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 -- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c0039b0 msgr2=0x7fd86c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:38.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c0039b0 0x7fd86c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd886000640 1 -- 172.21.15.123:0/2692993273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c003520 con 0x7fd86c004800 2024-04-25T19:03:38.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.137+0000 7fd887400640 1 -- 172.21.15.123:0/2692993273 reap_dead start 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.138+0000 7fd886000640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 0x7fd86c004c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c210 tx=0x7fd86c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9638c58df9832aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.138+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874007d10 con 0x7fd86c004800 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.138+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8740058d0 con 0x7fd86c004800 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.138+0000 7fd885600640 1 -- 172.21.15.123:0/2692993273 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874005d40 con 0x7fd86c004800 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 -- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 msgr2=0x7fd86c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 0x7fd86c004c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c210 tx=0x7fd86c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 -- 172.21.15.123:0/2692993273 shutdown_connections 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 --2- 172.21.15.123:0/2692993273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c004800 0x7fd86c004c10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 -- 172.21.15.123:0/2692993273 >> 172.21.15.123:0/2692993273 conn(0x7fd8880d65d0 msgr2=0x7fd8880b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 -- 172.21.15.123:0/2692993273 shutdown_connections 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.139+0000 7fd887e00640 1 -- 172.21.15.123:0/2692993273 wait complete. 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.142+0000 7fd887e00640 1 Processor -- start 2024-04-25T19:03:38.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.142+0000 7fd887e00640 1 -- start start 2024-04-25T19:03:38.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 0x7fd8881f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8880cc9c0 0x7fd888148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d4d40 con 0x7fd888159b40 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d1410 con 0x7fd86c004800 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.143+0000 7fd887e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880d4ec0 con 0x7fd8880cc9c0 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52090/0 (socket says 172.21.15.123:52090) 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 -- 172.21.15.123:0/2583447377 learned_addr learned my addr 172.21.15.123:0/2583447377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd886a00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 0x7fd8881f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd886000640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8880cc9c0 0x7fd888148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 -- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8880cc9c0 msgr2=0x7fd888148c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8880cc9c0 0x7fd888148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 -- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 msgr2=0x7fd8881f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 0x7fd8881f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 -- 172.21.15.123:0/2583447377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88815a3c0 con 0x7fd888159b40 2024-04-25T19:03:38.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd886a00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 0x7fd8881f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd887400640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8880d1b70 tx=0x7fd88000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.144+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880018070 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.145+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880005d60 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.145+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000a7f0 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.145+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881f63b0 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.145+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd888147e60 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.146+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888050740 con 0x7fd888159b40 2024-04-25T19:03:38.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.147+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd88000a990 con 0x7fd888159b40 2024-04-25T19:03:38.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.148+0000 7fd87ea00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 0x7fd8680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.148+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fd8800948b0 con 0x7fd888159b40 2024-04-25T19:03:38.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.148+0000 7fd886a00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 0x7fd8680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.149+0000 7fd886a00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 0x7fd8680769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd878006fd0 tx=0x7fd878008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.152+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8881f63b0 con 0x7fd888159b40 2024-04-25T19:03:38.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.351+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8880b87f0 con 0x7fd888159b40 2024-04-25T19:03:38.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.354+0000 7fd87ea00640 1 -- 172.21.15.123:0/2583447377 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7fd88815a3c0 con 0x7fd888159b40 2024-04-25T19:03:38.356 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:38.356 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.356+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 msgr2=0x7fd8680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.356+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 0x7fd8680769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd878006fd0 tx=0x7fd878008040 comp rx=0 tx=0).stop 2024-04-25T19:03:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.356+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 msgr2=0x7fd888149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.356+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8880d1b70 tx=0x7fd88000cb90 comp rx=0 tx=0).stop 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 shutdown_connections 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd8680744b0 0x7fd8680769c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888159b40 0x7fd888149160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8880cc9c0 0x7fd888148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 --2- 172.21.15.123:0/2583447377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd86c004800 0x7fd8881f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 >> 172.21.15.123:0/2583447377 conn(0x7fd8880d65d0 msgr2=0x7fd88815a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:38.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 shutdown_connections 2024-04-25T19:03:38.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.357+0000 7fd887e00640 1 -- 172.21.15.123:0/2583447377 wait complete. 2024-04-25T19:03:38.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:38.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:38.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.657+0000 7f978da00640 1 Processor -- start 2024-04-25T19:03:38.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.658+0000 7f978da00640 1 -- start start 2024-04-25T19:03:38.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.659+0000 7f978da00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97880d03a0 0x7f97880cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f978da00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97880cf300 con 0x7f97881610f0 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f978da00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97880cf5e0 con 0x7f9788070820 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f978da00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f97880cff10 con 0x7f97880d03a0 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97880d03a0 0x7f97880cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97880d03a0 0x7f97880cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52104/0 (socket says 172.21.15.123:52104) 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 -- 172.21.15.123:0/3974717373 learned_addr learned my addr 172.21.15.123:0/3974717373 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:38.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9787400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f97881610f0 0x7f978806f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49794/0 (socket says 172.21.15.123:49794) 2024-04-25T19:03:38.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 -- 172.21.15.123:0/3974717373 >> v1:172.21.15.146:6789/0 conn(0x7f9788070820 legacy=0x7f97881609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:38.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 -- 172.21.15.123:0/3974717373 >> v1:172.21.15.123:6789/0 conn(0x7f97881610f0 legacy=0x7f978806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.660+0000 7f9786a00640 1 -- 172.21.15.123:0/3974717373 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97880cf8c0 con 0x7f97880d03a0 2024-04-25T19:03:38.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.662+0000 7f9786a00640 1 --2- 172.21.15.123:0/3974717373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880cbd80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f977800d150 tx=0x7f977802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bf5c087d6aa8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.662+0000 7f9785600640 1 -- 172.21.15.123:0/3974717373 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778009600 con 0x7f97880d03a0 2024-04-25T19:03:38.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.662+0000 7f9785600640 1 -- 172.21.15.123:0/3974717373 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9778037070 con 0x7f97880d03a0 2024-04-25T19:03:38.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.662+0000 7f9785600640 1 -- 172.21.15.123:0/3974717373 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780099c0 con 0x7f97880d03a0 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 -- 172.21.15.123:0/3974717373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 msgr2=0x7f97880cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 --2- 172.21.15.123:0/3974717373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880cbd80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f977800d150 tx=0x7f977802c970 comp rx=0 tx=0).stop 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 -- 172.21.15.123:0/3974717373 shutdown_connections 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 --2- 172.21.15.123:0/3974717373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880cbd80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 -- 172.21.15.123:0/3974717373 >> 172.21.15.123:0/3974717373 conn(0x7f97880ce390 msgr2=0x7f97880ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 -- 172.21.15.123:0/3974717373 shutdown_connections 2024-04-25T19:03:38.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.663+0000 7f978da00640 1 -- 172.21.15.123:0/3974717373 wait complete. 2024-04-25T19:03:38.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.665+0000 7f978da00640 1 Processor -- start 2024-04-25T19:03:38.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.666+0000 7f978da00640 1 -- start start 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9788070820 0x7f978814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97880b53c0 0x7f97880af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97880cdca0 con 0x7f97880d03a0 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97880cc2c0 con 0x7f97880b53c0 2024-04-25T19:03:38.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f978da00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97880cde20 con 0x7f9788070820 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f9786000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f9786000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52112/0 (socket says 172.21.15.123:52112) 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.667+0000 7f9786000640 1 -- 172.21.15.123:0/365449979 learned_addr learned my addr 172.21.15.123:0/365449979 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786a00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9788070820 0x7f978814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 -- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9788070820 msgr2=0x7f978814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9788070820 0x7f978814a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 -- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97880b53c0 msgr2=0x7f97880af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97880b53c0 0x7f97880af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 -- 172.21.15.123:0/365449979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978814a9f0 con 0x7f97880d03a0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f9786000640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f977000b3f0 tx=0x7f977000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770015020 con 0x7f97880d03a0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9770011830 con 0x7f97880d03a0 2024-04-25T19:03:38.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770011c20 con 0x7f97880d03a0 2024-04-25T19:03:38.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.668+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97880b5ba0 con 0x7f97880d03a0 2024-04-25T19:03:38.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.669+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9788159aa0 con 0x7f97880d03a0 2024-04-25T19:03:38.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.670+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97880b5910 con 0x7f97880d03a0 2024-04-25T19:03:38.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.671+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9770018020 con 0x7f97880d03a0 2024-04-25T19:03:38.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.672+0000 7f977ea00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 0x7f97640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:38.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.672+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f9770095280 con 0x7f97880d03a0 2024-04-25T19:03:38.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.672+0000 7f9786a00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 0x7f97640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:38.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.673+0000 7f9786a00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 0x7f97640769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9778036000 tx=0x7f977800b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:38.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.677+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f977001f030 con 0x7f97880d03a0 2024-04-25T19:03:38.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.859+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9788076a40 con 0x7f97880d03a0 2024-04-25T19:03:38.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.861+0000 7f977ea00640 1 -- 172.21.15.123:0/365449979 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f977005f690 con 0x7f97880d03a0 2024-04-25T19:03:38.862 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 msgr2=0x7f97640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 0x7f97640769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9778036000 tx=0x7f977800b000 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 msgr2=0x7f97880af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f977000b3f0 tx=0x7f977000b8b0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 shutdown_connections 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97640744b0 0x7f97640769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97880b53c0 0x7f97880af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97880d03a0 0x7f97880af4b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 --2- 172.21.15.123:0/365449979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9788070820 0x7f978814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:38.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 >> 172.21.15.123:0/365449979 conn(0x7f97880ce390 msgr2=0x7f978815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:38.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 shutdown_connections 2024-04-25T19:03:38.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:38.863+0000 7f978da00640 1 -- 172.21.15.123:0/365449979 wait complete. 2024-04-25T19:03:38.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.163+0000 7f3ba1800640 1 Processor -- start 2024-04-25T19:03:39.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.164+0000 7f3ba1800640 1 -- start start 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba1800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3b9c15df10 0x7f3b9c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba1800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c0d1430 con 0x7f3b9c06d580 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba1800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c070980 con 0x7f3b9c0b0700 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba1800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c070f40 con 0x7f3b9c15df10 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3b9be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3b9c15df10 0x7f3b9c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba0e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3b9c06d580 0x7f3b9c06d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49804/0 (socket says 172.21.15.123:49804) 2024-04-25T19:03:39.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.166+0000 7f3ba0e00640 1 -- 172.21.15.123:0/878924343 learned_addr learned my addr 172.21.15.123:0/878924343 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:39.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.167+0000 7f3b9be00640 1 -- 172.21.15.123:0/878924343 >> v1:172.21.15.146:6789/0 conn(0x7f3b9c0b0700 legacy=0x7f3b9c162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.167+0000 7f3b9be00640 1 -- 172.21.15.123:0/878924343 >> v1:172.21.15.123:6789/0 conn(0x7f3b9c06d580 legacy=0x7f3b9c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.167+0000 7f3b9b400640 1 -- 172.21.15.123:0/878924343 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1012478237 0 0) 0x7f3b9c0d1430 con 0x7f3b9c06d580 2024-04-25T19:03:39.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.167+0000 7f3b9be00640 1 -- 172.21.15.123:0/878924343 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c070c60 con 0x7f3b9c15df10 2024-04-25T19:03:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.168+0000 7f3b9be00640 1 --2- 172.21.15.123:0/878924343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c15df10 0x7f3b9c0702f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0097a0 tx=0x7f3b8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8486ce404a84b1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.168+0000 7f3b9b400640 1 -- 172.21.15.123:0/878924343 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c037070 con 0x7f3b9c15df10 2024-04-25T19:03:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.168+0000 7f3b9b400640 1 -- 172.21.15.123:0/878924343 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c032440 con 0x7f3b9c15df10 2024-04-25T19:03:39.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.168+0000 7f3b9b400640 1 -- 172.21.15.123:0/878924343 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c032810 con 0x7f3b9c15df10 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 -- 172.21.15.123:0/878924343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c15df10 msgr2=0x7f3b9c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 --2- 172.21.15.123:0/878924343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c15df10 0x7f3b9c0702f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0097a0 tx=0x7f3b8c02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 -- 172.21.15.123:0/878924343 shutdown_connections 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 --2- 172.21.15.123:0/878924343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c15df10 0x7f3b9c0702f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 -- 172.21.15.123:0/878924343 >> 172.21.15.123:0/878924343 conn(0x7f3b9c073090 msgr2=0x7f3b9c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 -- 172.21.15.123:0/878924343 shutdown_connections 2024-04-25T19:03:39.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.169+0000 7f3ba1800640 1 -- 172.21.15.123:0/878924343 wait complete. 2024-04-25T19:03:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.171+0000 7f3ba1800640 1 Processor -- start 2024-04-25T19:03:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.172+0000 7f3ba1800640 1 -- start start 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b9c06d580 0x7f3b9c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b9c15df10 0x7f3b9c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cdfe0 con 0x7f3b9c0b0700 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c161e90 con 0x7f3b9c06d580 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3ba1800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0ce160 con 0x7f3b9c15df10 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3b93400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3b93400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52124/0 (socket says 172.21.15.123:52124) 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.173+0000 7f3b93400640 1 -- 172.21.15.123:0/141550000 learned_addr learned my addr 172.21.15.123:0/141550000 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b9be00640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b9c06d580 0x7f3b9c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3ba0e00640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b9c15df10 0x7f3b9c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 -- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b9c15df10 msgr2=0x7f3b9c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b9c15df10 0x7f3b9c0c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 -- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b9c06d580 msgr2=0x7f3b9c0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b9c06d580 0x7f3b9c0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 -- 172.21.15.123:0/141550000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c0aea40 con 0x7f3b9c0b0700 2024-04-25T19:03:39.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.174+0000 7f3b93400640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b420 tx=0x7f3b8400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.175+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84011830 con 0x7f3b9c0b0700 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.175+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b84016070 con 0x7f3b9c0b0700 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.175+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c0aed20 con 0x7f3b9c0b0700 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.175+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84011c10 con 0x7f3b9c0b0700 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.175+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c0af2e0 con 0x7f3b9c0b0700 2024-04-25T19:03:39.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.176+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c0ae7b0 con 0x7f3b9c0b0700 2024-04-25T19:03:39.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.177+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b8401b460 con 0x7f3b9c0b0700 2024-04-25T19:03:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.178+0000 7f3b98c00640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 0x7f3b780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.178+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f3b840950a0 con 0x7f3b9c0b0700 2024-04-25T19:03:39.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.178+0000 7f3b9be00640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 0x7f3b780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.179+0000 7f3b9be00640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 0x7f3b780769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c034000 tx=0x7f3b8c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.183+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b84020020 con 0x7f3b9c0b0700 2024-04-25T19:03:39.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.381+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b9c050740 con 0x7f3b9c0b0700 2024-04-25T19:03:39.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.382+0000 7f3b98c00640 1 -- 172.21.15.123:0/141550000 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f3b8405f4b0 con 0x7f3b9c0b0700 2024-04-25T19:03:39.384 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 msgr2=0x7f3b780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 0x7f3b780769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c034000 tx=0x7f3b8c02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 msgr2=0x7f3b9c1511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b420 tx=0x7f3b8400b8e0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 shutdown_connections 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3b780744e0 0x7f3b780769f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b9c15df10 0x7f3b9c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b9c0b0700 0x7f3b9c1511b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 --2- 172.21.15.123:0/141550000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b9c06d580 0x7f3b9c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 >> 172.21.15.123:0/141550000 conn(0x7f3b9c073090 msgr2=0x7f3b9c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 shutdown_connections 2024-04-25T19:03:39.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.385+0000 7f3ba1800640 1 -- 172.21.15.123:0/141550000 wait complete. 2024-04-25T19:03:39.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:39.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.681+0000 7f8e57e00640 1 Processor -- start 2024-04-25T19:03:39.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.681+0000 7f8e57e00640 1 -- start start 2024-04-25T19:03:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.683+0000 7f8e57e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e580b7c20 0x7f8e580d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.683+0000 7f8e57e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e580b17e0 con 0x7f8e580ce7c0 2024-04-25T19:03:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.683+0000 7f8e57e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e580b1ac0 con 0x7f8e580d18f0 2024-04-25T19:03:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.683+0000 7f8e57e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e580b4110 con 0x7f8e580b7c20 2024-04-25T19:03:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.684+0000 7f8e56a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8e580ce7c0 0x7f8e580b3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49812/0 (socket says 172.21.15.123:49812) 2024-04-25T19:03:39.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.684+0000 7f8e56a00640 1 -- 172.21.15.123:0/3955374113 learned_addr learned my addr 172.21.15.123:0/3955374113 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.684+0000 7f8e57400640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e580b7c20 0x7f8e580d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.684+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160183439 0 0) 0x7f8e580b17e0 con 0x7f8e580ce7c0 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.684+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e3c003520 con 0x7f8e580ce7c0 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 429436496 0 0) 0x7f8e3c003520 con 0x7f8e580ce7c0 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e580b7c20 msgr2=0x7f8e580d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e580b7c20 0x7f8e580d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 >> v1:172.21.15.146:6789/0 conn(0x7f8e580d18f0 legacy=0x7f8e580b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e580b1da0 con 0x7f8e580ce7c0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f8e48001670 con 0x7f8e580ce7c0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 >> v1:172.21.15.123:6789/0 conn(0x7f8e580ce7c0 legacy=0x7f8e580b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c0039e0 0x7f8e3c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c004350 0x7f8e3c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e3c005080 0x7f8e3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c00bf30 con 0x7f8e3c004350 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c00c0e0 con 0x7f8e3c005080 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c00c2e0 con 0x7f8e3c0039e0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.685+0000 7f8e56000640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c004350 0x7f8e3c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56a00640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c0039e0 0x7f8e3c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e57400640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e3c005080 0x7f8e3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56000640 1 -- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c0039e0 msgr2=0x7f8e3c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56000640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c0039e0 0x7f8e3c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56000640 1 -- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e3c005080 msgr2=0x7f8e3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56000640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e3c005080 0x7f8e3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e56000640 1 -- 172.21.15.123:0/3955374113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c003520 con 0x7f8e3c004350 2024-04-25T19:03:39.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.686+0000 7f8e57400640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e3c005080 0x7f8e3c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.687+0000 7f8e56000640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c004350 0x7f8e3c004880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003db0 tx=0x7f8e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b622e1887f93081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.687+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44037070 con 0x7f8e3c004350 2024-04-25T19:03:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.687+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4402aa00 con 0x7f8e3c004350 2024-04-25T19:03:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.687+0000 7f8e55600640 1 -- 172.21.15.123:0/3955374113 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44003980 con 0x7f8e3c004350 2024-04-25T19:03:39.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 -- 172.21.15.123:0/3955374113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c004350 msgr2=0x7f8e3c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 --2- 172.21.15.123:0/3955374113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c004350 0x7f8e3c004880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003db0 tx=0x7f8e3c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57400640 1 -- 172.21.15.123:0/3955374113 reap_dead start 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 -- 172.21.15.123:0/3955374113 shutdown_connections 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 -- 172.21.15.123:0/3955374113 >> 172.21.15.123:0/3955374113 conn(0x7f8e5809b4a0 msgr2=0x7f8e5809b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 -- 172.21.15.123:0/3955374113 shutdown_connections 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.688+0000 7f8e57e00640 1 -- 172.21.15.123:0/3955374113 wait complete. 2024-04-25T19:03:39.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.690+0000 7f8e57e00640 1 Processor -- start 2024-04-25T19:03:39.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.691+0000 7f8e57e00640 1 -- start start 2024-04-25T19:03:39.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c004350 0x7f8e5809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 0x7f8e580c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580cff90 con 0x7f8e58160030 2024-04-25T19:03:39.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580cfcb0 con 0x7f8e5815fc20 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e580cef20 con 0x7f8e3c004350 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52150/0 (socket says 172.21.15.123:52150) 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.692+0000 7f8e57400640 1 -- 172.21.15.123:0/1693168017 learned_addr learned my addr 172.21.15.123:0/1693168017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e56a00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c004350 0x7f8e5809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e56000640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 0x7f8e580c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 -- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c004350 msgr2=0x7f8e5809da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c004350 0x7f8e5809da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 -- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 msgr2=0x7f8e580c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 0x7f8e580c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 -- 172.21.15.123:0/1693168017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5809df40 con 0x7f8e58160030 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e56000640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 0x7f8e580c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:39.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e57400640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e580738f0 tx=0x7f8e5000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5001c070 con 0x7f8e58160030 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.693+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e50017400 con 0x7f8e58160030 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.694+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5809c960 con 0x7f8e58160030 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.694+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500177d0 con 0x7f8e58160030 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.694+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5809cf20 con 0x7f8e58160030 2024-04-25T19:03:39.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.695+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e580d0f70 con 0x7f8e58160030 2024-04-25T19:03:39.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.696+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e50005d10 con 0x7f8e58160030 2024-04-25T19:03:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.697+0000 7f8e4ea00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.697+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f8e50095880 con 0x7f8e58160030 2024-04-25T19:03:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.697+0000 7f8e56a00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.698+0000 7f8e56a00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 0x7f8e380769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e48001050 tx=0x7f8e48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.702+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e580d0f70 con 0x7f8e58160030 2024-04-25T19:03:39.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.903+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e58050740 con 0x7f8e58160030 2024-04-25T19:03:39.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.903+0000 7f8e4ea00640 1 -- 172.21.15.123:0/1693168017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f8e50012070 con 0x7f8e58160030 2024-04-25T19:03:39.905 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 msgr2=0x7f8e380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 0x7f8e380769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e48001050 tx=0x7f8e48009040 comp rx=0 tx=0).stop 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 msgr2=0x7f8e5809fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e580738f0 tx=0x7f8e5000f4a0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 shutdown_connections 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8e380744b0 0x7f8e380769c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58160030 0x7f8e5809fca0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e5815fc20 0x7f8e580c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 --2- 172.21.15.123:0/1693168017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e3c004350 0x7f8e5809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 >> 172.21.15.123:0/1693168017 conn(0x7f8e5809b4a0 msgr2=0x7f8e580b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 shutdown_connections 2024-04-25T19:03:39.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:39.906+0000 7f8e57e00640 1 -- 172.21.15.123:0/1693168017 wait complete. 2024-04-25T19:03:39.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:40.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.211+0000 7fbee2200640 1 Processor -- start 2024-04-25T19:03:40.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.211+0000 7fbee2200640 1 -- start start 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbee2200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbedc0cd140 0x7fbedc15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbee2200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 0x7fbedc0af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbee2200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbedc0cc5f0 con 0x7fbedc06daa0 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbee2200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbedc06deb0 con 0x7fbedc0cd140 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbee2200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbedc06d610 con 0x7fbedc162540 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbed3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbedc0cd140 0x7fbedc15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbedb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbedc06daa0 0x7fbedc0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49820/0 (socket says 172.21.15.123:49820) 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.213+0000 7fbedb400640 1 -- 172.21.15.123:0/2498657840 learned_addr learned my addr 172.21.15.123:0/2498657840 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbedaa00640 1 -- 172.21.15.123:0/2498657840 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431246553 0 0) 0x7fbedc0cc5f0 con 0x7fbedc06daa0 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbedaa00640 1 -- 172.21.15.123:0/2498657840 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbec0003520 con 0x7fbedc06daa0 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbed3e00640 1 -- 172.21.15.123:0/2498657840 >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 msgr2=0x7fbedc0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbedbe00640 1 --2- 172.21.15.123:0/2498657840 >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 0x7fbedc0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbed3e00640 1 --2- 172.21.15.123:0/2498657840 >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 0x7fbedc0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbed3e00640 1 -- 172.21.15.123:0/2498657840 >> v1:172.21.15.123:6789/0 conn(0x7fbedc06daa0 legacy=0x7fbedc0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbed3e00640 1 -- 172.21.15.123:0/2498657840 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc06ca00 con 0x7fbedc0cd140 2024-04-25T19:03:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.214+0000 7fbedbe00640 1 --2- 172.21.15.123:0/2498657840 >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 0x7fbedc0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:40.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.215+0000 7fbed3e00640 1 --2- 172.21.15.123:0/2498657840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc0cd140 0x7fbedc15dfc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0d01c0 tx=0x7fbec402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eafd043769d9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbedaa00640 1 -- 172.21.15.123:0/2498657840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4037070 con 0x7fbedc0cd140 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbedaa00640 1 -- 172.21.15.123:0/2498657840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec4032440 con 0x7fbedc0cd140 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbedaa00640 1 -- 172.21.15.123:0/2498657840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4032810 con 0x7fbedc0cd140 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 -- 172.21.15.123:0/2498657840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc0cd140 msgr2=0x7fbedc15dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 --2- 172.21.15.123:0/2498657840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc0cd140 0x7fbedc15dfc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0d01c0 tx=0x7fbec402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 -- 172.21.15.123:0/2498657840 shutdown_connections 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 --2- 172.21.15.123:0/2498657840 >> v2:172.21.15.146:3300/0 conn(0x7fbedc162540 0x7fbedc0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 --2- 172.21.15.123:0/2498657840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc0cd140 0x7fbedc15dfc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.216+0000 7fbee2200640 1 -- 172.21.15.123:0/2498657840 >> 172.21.15.123:0/2498657840 conn(0x7fbedc0ce390 msgr2=0x7fbedc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.217+0000 7fbee2200640 1 -- 172.21.15.123:0/2498657840 shutdown_connections 2024-04-25T19:03:40.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.217+0000 7fbee2200640 1 -- 172.21.15.123:0/2498657840 wait complete. 2024-04-25T19:03:40.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.219+0000 7fbee2200640 1 Processor -- start 2024-04-25T19:03:40.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.219+0000 7fbee2200640 1 -- start start 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbedc06daa0 0x7fbedc1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbedc162540 0x7fbedc1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc06d2a0 con 0x7fbedc1585b0 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc06cfc0 con 0x7fbedc162540 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbee2200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbedc162950 con 0x7fbedc06daa0 2024-04-25T19:03:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbedbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbedbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52168/0 (socket says 172.21.15.123:52168) 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbedbe00640 1 -- 172.21.15.123:0/4255302883 learned_addr learned my addr 172.21.15.123:0/4255302883 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbed3e00640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbedc162540 0x7fbedc1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.221+0000 7fbedb400640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbedc06daa0 0x7fbedc1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 -- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbedc06daa0 msgr2=0x7fbedc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbedc06daa0 0x7fbedc1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 -- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbedc162540 msgr2=0x7fbedc1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbedc162540 0x7fbedc1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 -- 172.21.15.123:0/4255302883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc1f9290 con 0x7fbedc1585b0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbedbe00640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cd80 tx=0x7fbed400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed400f680 con 0x7fbedc1585b0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed400f840 con 0x7fbedc1585b0 2024-04-25T19:03:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbedc1f9570 con 0x7fbedc1585b0 2024-04-25T19:03:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed400fc30 con 0x7fbedc1585b0 2024-04-25T19:03:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.222+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbedc1f9b30 con 0x7fbedc1585b0 2024-04-25T19:03:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.223+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbedc076a40 con 0x7fbedc1585b0 2024-04-25T19:03:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.225+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbed40213f0 con 0x7fbedc1585b0 2024-04-25T19:03:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.226+0000 7fbed3400640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 0x7fbeb80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.226+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fbed4096240 con 0x7fbedc1585b0 2024-04-25T19:03:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.226+0000 7fbedb400640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 0x7fbeb80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.227+0000 7fbedb400640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 0x7fbeb80769f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005880 tx=0x7fbecc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.230+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbed4014030 con 0x7fbedc1585b0 2024-04-25T19:03:40.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.425+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbedc069c60 con 0x7fbedc1585b0 2024-04-25T19:03:40.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.426+0000 7fbed3400640 1 -- 172.21.15.123:0/4255302883 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fbed40606f0 con 0x7fbedc1585b0 2024-04-25T19:03:40.428 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:40.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 msgr2=0x7fbeb80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 0x7fbeb80769f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005880 tx=0x7fbecc009040 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 msgr2=0x7fbedc0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cd80 tx=0x7fbed400a650 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 shutdown_connections 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbeb80744e0 0x7fbeb80769f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbedc1585b0 0x7fbedc0d0540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbedc162540 0x7fbedc1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 --2- 172.21.15.123:0/4255302883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbedc06daa0 0x7fbedc1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.428+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 >> 172.21.15.123:0/4255302883 conn(0x7fbedc0ce390 msgr2=0x7fbedc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.429+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 shutdown_connections 2024-04-25T19:03:40.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.429+0000 7fbee2200640 1 -- 172.21.15.123:0/4255302883 wait complete. 2024-04-25T19:03:40.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:40.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.706+0000 7f2b25c00640 1 Processor -- start 2024-04-25T19:03:40.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.706+0000 7f2b25c00640 1 -- start start 2024-04-25T19:03:40.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.708+0000 7f2b25c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b2015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.708+0000 7f2b25c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b2015f070 con 0x7f2b2006fe20 2024-04-25T19:03:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.708+0000 7f2b25c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b2006f340 con 0x7f2b2006d580 2024-04-25T19:03:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.708+0000 7f2b25c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2006f620 con 0x7f2b2015e2f0 2024-04-25T19:03:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2b2006fe20 0x7f2b2015b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49826/0 (socket says 172.21.15.123:49826) 2024-04-25T19:03:40.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1fe00640 1 -- 172.21.15.123:0/80689670 learned_addr learned my addr 172.21.15.123:0/80689670 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:40.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b25200640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b2015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243391231 0 0) 0x7f2b2015f070 con 0x7f2b2006fe20 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b04003520 con 0x7f2b2006fe20 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3695899311 0 0) 0x7f2b04003520 con 0x7f2b2006fe20 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 msgr2=0x7f2b2015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b2015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 >> v1:172.21.15.146:6789/0 conn(0x7f2b2006d580 legacy=0x7f2b2006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.709+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2006f900 con 0x7f2b2006fe20 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f2b10001670 con 0x7f2b2006fe20 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b25200640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b2015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 >> v1:172.21.15.123:6789/0 conn(0x7f2b2006fe20 legacy=0x7f2b2015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b040039e0 0x7f2b04003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b04004d40 0x7f2b04005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006f620 con 0x7f2b040039e0 2024-04-25T19:03:40.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0400bc40 con 0x7f2b04004d40 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0400be40 con 0x7f2b2015e2f0 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b040039e0 0x7f2b04003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b25200640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b04004d40 0x7f2b04005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 -- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 msgr2=0x7f2b04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b2015e2f0 0x7f2b04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 -- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b04004d40 msgr2=0x7f2b04005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b04004d40 0x7f2b04005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.710+0000 7f2b1fe00640 1 -- 172.21.15.123:0/80689670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b04003520 con 0x7f2b040039e0 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.711+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b040039e0 0x7f2b04003e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b20070d80 tx=0x7f2b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9396539c9a478d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10007c20 con 0x7f2b040039e0 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b10005af0 con 0x7f2b040039e0 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b1ea00640 1 -- 172.21.15.123:0/80689670 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10007070 con 0x7f2b040039e0 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b25c00640 1 -- 172.21.15.123:0/80689670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b040039e0 msgr2=0x7f2b04003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b25c00640 1 --2- 172.21.15.123:0/80689670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b040039e0 0x7f2b04003e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b20070d80 tx=0x7f2b04005830 comp rx=0 tx=0).stop 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b25200640 1 -- 172.21.15.123:0/80689670 reap_dead start 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b25c00640 1 -- 172.21.15.123:0/80689670 shutdown_connections 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.712+0000 7f2b25c00640 1 -- 172.21.15.123:0/80689670 >> 172.21.15.123:0/80689670 conn(0x7f2b20073200 msgr2=0x7f2b20073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.713+0000 7f2b25c00640 1 -- 172.21.15.123:0/80689670 shutdown_connections 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.713+0000 7f2b25c00640 1 -- 172.21.15.123:0/80689670 wait complete. 2024-04-25T19:03:40.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.715+0000 7f2b25c00640 1 Processor -- start 2024-04-25T19:03:40.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.715+0000 7f2b25c00640 1 -- start start 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b040039e0 0x7f2b201f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 0x7f2b2013d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2015ea90 con 0x7f2b2013a470 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2015e910 con 0x7f2b200cf290 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2015ec10 con 0x7f2b040039e0 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52194/0 (socket says 172.21.15.123:52194) 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 -- 172.21.15.123:0/3238545174 learned_addr learned my addr 172.21.15.123:0/3238545174 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b040039e0 0x7f2b201f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25200640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 0x7f2b2013d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 -- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b040039e0 msgr2=0x7f2b201f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b040039e0 0x7f2b201f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 -- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 msgr2=0x7f2b2013d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 0x7f2b2013d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b1f400640 1 -- 172.21.15.123:0/3238545174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b200cd440 con 0x7f2b2013a470 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.717+0000 7f2b25200640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 0x7f2b2013d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b1f400640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c500 tx=0x7f2b0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0052d0 con 0x7f2b2013a470 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0c005470 con 0x7f2b2013a470 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c012850 con 0x7f2b2013a470 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b200cbe60 con 0x7f2b2013a470 2024-04-25T19:03:40.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.718+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b200cc420 con 0x7f2b2013a470 2024-04-25T19:03:40.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.719+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b200cbbd0 con 0x7f2b2013a470 2024-04-25T19:03:40.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.720+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b0c0129f0 con 0x7f2b2013a470 2024-04-25T19:03:40.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.722+0000 7f2b17e00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 0x7f2b000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:40.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.722+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f2b0c0952a0 con 0x7f2b2013a470 2024-04-25T19:03:40.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.722+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 0x7f2b000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:40.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.723+0000 7f2b1fe00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 0x7f2b000769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b10032000 tx=0x7f2b10009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:40.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.727+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b200cd440 con 0x7f2b2013a470 2024-04-25T19:03:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.924+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b20050740 con 0x7f2b2013a470 2024-04-25T19:03:40.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.927+0000 7f2b17e00640 1 -- 172.21.15.123:0/3238545174 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f2b0c05f740 con 0x7f2b2013a470 2024-04-25T19:03:40.930 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:40.930 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.929+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 msgr2=0x7f2b000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.929+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 0x7f2b000769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b10032000 tx=0x7f2b10009030 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.929+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 msgr2=0x7f2b200ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.929+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c500 tx=0x7f2b0c00c9c0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 shutdown_connections 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2b000744b0 0x7f2b000769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b200cf290 0x7f2b2013d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2013a470 0x7f2b200ccf00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 --2- 172.21.15.123:0/3238545174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b040039e0 0x7f2b201f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 >> 172.21.15.123:0/3238545174 conn(0x7f2b20073200 msgr2=0x7f2b20160c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 shutdown_connections 2024-04-25T19:03:40.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:40.930+0000 7f2b25c00640 1 -- 172.21.15.123:0/3238545174 wait complete. 2024-04-25T19:03:40.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:40.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:41.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.230+0000 7fda5a600640 1 Processor -- start 2024-04-25T19:03:41.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.230+0000 7fda5a600640 1 -- start start 2024-04-25T19:03:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.232+0000 7fda5a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.232+0000 7fda5a600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda54162540 0x7fda540af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.232+0000 7fda5a600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda540cc5f0 con 0x7fda5406daa0 2024-04-25T19:03:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.232+0000 7fda5a600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fda5406deb0 con 0x7fda54162540 2024-04-25T19:03:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.232+0000 7fda5a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda5406d610 con 0x7fda540cd140 2024-04-25T19:03:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.233+0000 7fda52a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.233+0000 7fda53e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda54162540 0x7fda540af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.233+0000 7fda52a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:43476/0 (socket says 172.21.15.123:43476) 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.233+0000 7fda52a00640 1 -- 172.21.15.123:0/3922033425 learned_addr learned my addr 172.21.15.123:0/3922033425 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.233+0000 7fda53400640 1 --1- 172.21.15.123:0/3922033425 >> v1:172.21.15.146:6789/0 conn(0x7fda5406daa0 0x7fda540cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:46558/0 (socket says 172.21.15.123:46558) 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.234+0000 7fda52a00640 1 -- 172.21.15.123:0/3922033425 >> v2:172.21.15.146:3300/0 conn(0x7fda54162540 msgr2=0x7fda540af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.234+0000 7fda52a00640 1 --2- 172.21.15.123:0/3922033425 >> v2:172.21.15.146:3300/0 conn(0x7fda54162540 0x7fda540af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.234+0000 7fda52a00640 1 -- 172.21.15.123:0/3922033425 >> v1:172.21.15.146:6789/0 conn(0x7fda5406daa0 legacy=0x7fda540cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.234+0000 7fda52a00640 1 -- 172.21.15.123:0/3922033425 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5406ca00 con 0x7fda540cd140 2024-04-25T19:03:41.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.235+0000 7fda52a00640 1 --2- 172.21.15.123:0/3922033425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009870 tx=0x7fda3c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2e1d74fe692a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.236+0000 7fda52000640 1 -- 172.21.15.123:0/3922033425 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c037070 con 0x7fda540cd140 2024-04-25T19:03:41.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.236+0000 7fda52000640 1 -- 172.21.15.123:0/3922033425 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c032440 con 0x7fda540cd140 2024-04-25T19:03:41.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.236+0000 7fda52000640 1 -- 172.21.15.123:0/3922033425 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c032810 con 0x7fda540cd140 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 -- 172.21.15.123:0/3922033425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 msgr2=0x7fda5415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 --2- 172.21.15.123:0/3922033425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009870 tx=0x7fda3c02a250 comp rx=0 tx=0).stop 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 -- 172.21.15.123:0/3922033425 shutdown_connections 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 --2- 172.21.15.123:0/3922033425 >> v2:172.21.15.146:3300/0 conn(0x7fda54162540 0x7fda540af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 --2- 172.21.15.123:0/3922033425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda540cd140 0x7fda5415dfc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 -- 172.21.15.123:0/3922033425 >> 172.21.15.123:0/3922033425 conn(0x7fda540ce390 msgr2=0x7fda540ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 -- 172.21.15.123:0/3922033425 shutdown_connections 2024-04-25T19:03:41.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.237+0000 7fda5a600640 1 -- 172.21.15.123:0/3922033425 wait complete. 2024-04-25T19:03:41.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.240+0000 7fda5a600640 1 Processor -- start 2024-04-25T19:03:41.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.241+0000 7fda5a600640 1 -- start start 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda54162540 0x7fda540b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 0x7fda5415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda5406d2a0 con 0x7fda5406daa0 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda5406cfc0 con 0x7fda54162540 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda5a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda54162950 con 0x7fda54157970 2024-04-25T19:03:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda53400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda53400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52206/0 (socket says 172.21.15.123:52206) 2024-04-25T19:03:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.242+0000 7fda53400640 1 -- 172.21.15.123:0/955965544 learned_addr learned my addr 172.21.15.123:0/955965544 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53e00640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 0x7fda5415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda52a00640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda54162540 0x7fda540b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 -- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 msgr2=0x7fda5415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 0x7fda5415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 -- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda54162540 msgr2=0x7fda540b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda54162540 0x7fda540b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 -- 172.21.15.123:0/955965544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54158150 con 0x7fda5406daa0 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53e00640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 0x7fda5415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda53400640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c3f0 tx=0x7fda4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.243+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440074e0 con 0x7fda5406daa0 2024-04-25T19:03:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.244+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda44007680 con 0x7fda5406daa0 2024-04-25T19:03:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.244+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda541f9f00 con 0x7fda5406daa0 2024-04-25T19:03:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.244+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440049e0 con 0x7fda5406daa0 2024-04-25T19:03:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.244+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda541fa1a0 con 0x7fda5406daa0 2024-04-25T19:03:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.245+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda540cc170 con 0x7fda5406daa0 2024-04-25T19:03:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.246+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda4401a020 con 0x7fda5406daa0 2024-04-25T19:03:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.247+0000 7fda4b400640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 0x7fda300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.247+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fda44094db0 con 0x7fda5406daa0 2024-04-25T19:03:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.247+0000 7fda52a00640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 0x7fda300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.248+0000 7fda52a00640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 0x7fda300769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda3c034000 tx=0x7fda3c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.252+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda540cc170 con 0x7fda5406daa0 2024-04-25T19:03:41.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.455+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda540cd720 con 0x7fda5406daa0 2024-04-25T19:03:41.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.456+0000 7fda4b400640 1 -- 172.21.15.123:0/955965544 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fda540cd720 con 0x7fda5406daa0 2024-04-25T19:03:41.458 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 msgr2=0x7fda300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 0x7fda300769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda3c034000 tx=0x7fda3c02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 msgr2=0x7fda540b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c3f0 tx=0x7fda4400c8b0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 shutdown_connections 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fda300744b0 0x7fda300769c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda54157970 0x7fda5415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda54162540 0x7fda540b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 --2- 172.21.15.123:0/955965544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda5406daa0 0x7fda540b6500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 >> 172.21.15.123:0/955965544 conn(0x7fda540ce390 msgr2=0x7fda540b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 shutdown_connections 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.459+0000 7fda5a600640 1 -- 172.21.15.123:0/955965544 wait complete. 2024-04-25T19:03:41.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.757+0000 7fead5e00640 1 Processor -- start 2024-04-25T19:03:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.757+0000 7fead5e00640 1 -- start start 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.759+0000 7fead5e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead006d580 0x7fead006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.759+0000 7fead5e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fead00b4f40 0x7fead00b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.759+0000 7fead5e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead006ded0 con 0x7fead014da70 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.759+0000 7fead5e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fead00b44f0 con 0x7fead00b4f40 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.759+0000 7fead5e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fead01653e0 con 0x7fead006d580 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead4a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fead014da70 0x7fead0164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49836/0 (socket says 172.21.15.123:49836) 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead4a00640 1 -- 172.21.15.123:0/3233943910 learned_addr learned my addr 172.21.15.123:0/3233943910 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead5400640 1 --2- 172.21.15.123:0/3233943910 >> v2:172.21.15.123:3300/0 conn(0x7fead00b4f40 0x7fead00b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7feacfe00640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead006d580 0x7fead006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7feacf400640 1 -- 172.21.15.123:0/3233943910 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672699907 0 0) 0x7feab8001110 con 0x7fead014da70 2024-04-25T19:03:41.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7feacf400640 1 -- 172.21.15.123:0/3233943910 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab4003520 con 0x7fead014da70 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead5400640 1 -- 172.21.15.123:0/3233943910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead006d580 msgr2=0x7fead006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead5400640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead006d580 0x7fead006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead5400640 1 -- 172.21.15.123:0/3233943910 >> v1:172.21.15.123:6789/0 conn(0x7fead014da70 legacy=0x7fead0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.760+0000 7fead5400640 1 -- 172.21.15.123:0/3233943910 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead01656c0 con 0x7fead00b4f40 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.761+0000 7fead5400640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead00b4f40 0x7fead00b5350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feac80097a0 tx=0x7feac802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97315717f3d6db91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.761+0000 7feacf400640 1 -- 172.21.15.123:0/3233943910 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8037070 con 0x7fead00b4f40 2024-04-25T19:03:41.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7feacf400640 1 -- 172.21.15.123:0/3233943910 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac8032440 con 0x7fead00b4f40 2024-04-25T19:03:41.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7feacf400640 1 -- 172.21.15.123:0/3233943910 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8032810 con 0x7fead00b4f40 2024-04-25T19:03:41.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 -- 172.21.15.123:0/3233943910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead00b4f40 msgr2=0x7fead00b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead00b4f40 0x7fead00b5350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feac80097a0 tx=0x7feac802a180 comp rx=0 tx=0).stop 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 -- 172.21.15.123:0/3233943910 shutdown_connections 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead00b4f40 0x7fead00b5350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 --2- 172.21.15.123:0/3233943910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead006d580 0x7fead006d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 -- 172.21.15.123:0/3233943910 >> 172.21.15.123:0/3233943910 conn(0x7fead0073090 msgr2=0x7fead00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 -- 172.21.15.123:0/3233943910 shutdown_connections 2024-04-25T19:03:41.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.762+0000 7fead5e00640 1 -- 172.21.15.123:0/3233943910 wait complete. 2024-04-25T19:03:41.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.765+0000 7fead5e00640 1 Processor -- start 2024-04-25T19:03:41.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.765+0000 7fead5e00640 1 -- start start 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead00b4f40 0x7fead015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fead014da70 0x7fead00b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fead014de80 con 0x7fead006d580 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fead00b4a20 con 0x7fead014da70 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.766+0000 7fead5e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fead0165030 con 0x7fead00b4f40 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52238/0 (socket says 172.21.15.123:52238) 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 -- 172.21.15.123:0/3185581458 learned_addr learned my addr 172.21.15.123:0/3185581458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7feacfe00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead00b4f40 0x7fead015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead5400640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fead014da70 0x7fead00b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 -- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead00b4f40 msgr2=0x7fead015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead00b4f40 0x7fead015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 -- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fead014da70 msgr2=0x7fead00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fead014da70 0x7fead00b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 -- 172.21.15.123:0/3185581458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead00cd260 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.767+0000 7fead4a00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feab800c3f0 tx=0x7feab800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.768+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8017070 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.768+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab8005ce0 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.768+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80072f0 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.768+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead015f920 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.768+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead015fee0 con 0x7fead006d580 2024-04-25T19:03:41.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.769+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead0050740 con 0x7fead006d580 2024-04-25T19:03:41.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.770+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feab801a070 con 0x7fead006d580 2024-04-25T19:03:41.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.771+0000 7feaccc00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 0x7feaac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.771+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7feab8094940 con 0x7fead006d580 2024-04-25T19:03:41.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.772+0000 7feacfe00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 0x7feaac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.772+0000 7feacfe00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 0x7feaac0769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fead01fd8f0 tx=0x7feac00039b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.777+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fead006ded0 con 0x7fead006d580 2024-04-25T19:03:41.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.981+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fead00742b0 con 0x7fead006d580 2024-04-25T19:03:41.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.982+0000 7feaccc00640 1 -- 172.21.15.123:0/3185581458 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7feab805ed50 con 0x7fead006d580 2024-04-25T19:03:41.984 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:41.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 msgr2=0x7feaac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 0x7feaac0769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fead01fd8f0 tx=0x7feac00039b0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 msgr2=0x7fead00ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feab800c3f0 tx=0x7feab800c8b0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 shutdown_connections 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7feaac0744b0 0x7feaac0769c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fead014da70 0x7fead00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fead00b4f40 0x7fead015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 --2- 172.21.15.123:0/3185581458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead006d580 0x7fead00ccd20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.985+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 >> 172.21.15.123:0/3185581458 conn(0x7fead0073090 msgr2=0x7fead0167a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.986+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 shutdown_connections 2024-04-25T19:03:41.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:41.986+0000 7fead5e00640 1 -- 172.21.15.123:0/3185581458 wait complete. 2024-04-25T19:03:42.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:42.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.288+0000 7fbd3ba00640 1 Processor -- start 2024-04-25T19:03:42.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.288+0000 7fbd3ba00640 1 -- start start 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd3ba00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd340cd310 0x7fbd340c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd3ba00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd340d0a10 0x7fbd34163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd3ba00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340c85b0 con 0x7fbd34164190 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd3ba00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbd34164b00 con 0x7fbd340cd310 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd3ba00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3406c570 con 0x7fbd340d0a10 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.290+0000 7fbd39a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbd34164190 0x7fbd34160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49850/0 (socket says 172.21.15.123:49850) 2024-04-25T19:03:42.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd39a00640 1 -- 172.21.15.123:0/1097205402 learned_addr learned my addr 172.21.15.123:0/1097205402 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:42.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd39000640 1 --2- 172.21.15.123:0/1097205402 >> v2:172.21.15.146:3300/0 conn(0x7fbd340cd310 0x7fbd340c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd33e00640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd340d0a10 0x7fbd34163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152423833 0 0) 0x7fbd340c85b0 con 0x7fbd34164190 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd18003520 con 0x7fbd34164190 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375239892 0 0) 0x7fbd18003520 con 0x7fbd34164190 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd39000640 1 -- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd340d0a10 msgr2=0x7fbd34163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd39000640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd340d0a10 0x7fbd34163c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.291+0000 7fbd39000640 1 -- 172.21.15.123:0/1097205402 >> v1:172.21.15.123:6789/0 conn(0x7fbd34164190 legacy=0x7fbd34160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.292+0000 7fbd39000640 1 -- 172.21.15.123:0/1097205402 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3415ab40 con 0x7fbd340cd310 2024-04-25T19:03:42.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.293+0000 7fbd39000640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd340c8070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009d90 tx=0x7fbd2402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce02538dbd70f334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.293+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c001670 con 0x7fbd340cd310 2024-04-25T19:03:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24007540 con 0x7fbd340cd310 2024-04-25T19:03:42.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd33400640 1 -- 172.21.15.123:0/1097205402 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240078f0 con 0x7fbd340cd310 2024-04-25T19:03:42.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 msgr2=0x7fbd340c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd340c8070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009d90 tx=0x7fbd2402a5f0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1097205402 shutdown_connections 2024-04-25T19:03:42.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd340d0a10 0x7fbd34163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1097205402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd340c8070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1097205402 >> 172.21.15.123:0/1097205402 conn(0x7fbd340ce150 msgr2=0x7fbd340ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.294+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1097205402 shutdown_connections 2024-04-25T19:03:42.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.295+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1097205402 wait complete. 2024-04-25T19:03:42.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.297+0000 7fbd3ba00640 1 Processor -- start 2024-04-25T19:03:42.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.297+0000 7fbd3ba00640 1 -- start start 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.298+0000 7fbd3ba00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd3414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd3ba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd3ba00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd34164190 0x7fbd340b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd3ba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341645a0 con 0x7fbd340d0a10 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd3ba00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406c850 con 0x7fbd340cd310 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd3ba00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd340cdb20 con 0x7fbd34164190 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52252/0 (socket says 172.21.15.123:52252) 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 -- 172.21.15.123:0/1786092427 learned_addr learned my addr 172.21.15.123:0/1786092427 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 -- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd34164190 msgr2=0x7fbd340b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd34164190 0x7fbd340b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 -- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 msgr2=0x7fbd3414a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd3414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.299+0000 7fbd33e00640 1 -- 172.21.15.123:0/1786092427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3414a690 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd33e00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e870 tx=0x7fbd1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c017070 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c00c8e0 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd340b4660 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00ccd0 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.300+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd340b4c20 con 0x7fbd340d0a10 2024-04-25T19:03:42.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.301+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34076a40 con 0x7fbd340d0a10 2024-04-25T19:03:42.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.303+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd1c01a070 con 0x7fbd340d0a10 2024-04-25T19:03:42.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.303+0000 7fbd30c00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 0x7fbd100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.304+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fbd1c095880 con 0x7fbd340d0a10 2024-04-25T19:03:42.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.304+0000 7fbd39000640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 0x7fbd100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.305+0000 7fbd39000640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 0x7fbd100769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd24034000 tx=0x7fbd2402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.308+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd1c05fc10 con 0x7fbd340d0a10 2024-04-25T19:03:42.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.505+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd340c7110 con 0x7fbd340d0a10 2024-04-25T19:03:42.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.506+0000 7fbd30c00640 1 -- 172.21.15.123:0/1786092427 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fbd1c05f510 con 0x7fbd340d0a10 2024-04-25T19:03:42.508 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 msgr2=0x7fbd100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 0x7fbd100769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd24034000 tx=0x7fbd2402e000 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 msgr2=0x7fbd341ea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e870 tx=0x7fbd1c00ed30 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 shutdown_connections 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd100744b0 0x7fbd100769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd34164190 0x7fbd340b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd340d0a10 0x7fbd341ea6f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 --2- 172.21.15.123:0/1786092427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd340cd310 0x7fbd3414a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 >> 172.21.15.123:0/1786092427 conn(0x7fbd340ce150 msgr2=0x7fbd3406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 shutdown_connections 2024-04-25T19:03:42.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.509+0000 7fbd3ba00640 1 -- 172.21.15.123:0/1786092427 wait complete. 2024-04-25T19:03:42.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:42.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.796+0000 7fc07bc00640 1 Processor -- start 2024-04-25T19:03:42.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.796+0000 7fc07bc00640 1 -- start start 2024-04-25T19:03:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.798+0000 7fc07bc00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0740cd140 0x7fc07415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.798+0000 7fc07bc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0740cc5f0 con 0x7fc07406daa0 2024-04-25T19:03:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.798+0000 7fc07bc00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07406ca00 con 0x7fc074162540 2024-04-25T19:03:42.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.798+0000 7fc07bc00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc07406deb0 con 0x7fc0740cd140 2024-04-25T19:03:42.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.799+0000 7fc079200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc07406daa0 0x7fc0740cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49856/0 (socket says 172.21.15.123:49856) 2024-04-25T19:03:42.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.799+0000 7fc079200640 1 -- 172.21.15.123:0/3417368490 learned_addr learned my addr 172.21.15.123:0/3417368490 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.799+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078834006 0 0) 0x7fc0740cc5f0 con 0x7fc07406daa0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.799+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc058003520 con 0x7fc07406daa0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.799+0000 7fc073e00640 1 --2- 172.21.15.123:0/3417368490 >> v2:172.21.15.146:3300/0 conn(0x7fc0740cd140 0x7fc07415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1257605019 0 0) 0x7fc058003520 con 0x7fc07406daa0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 >> v2:172.21.15.146:3300/0 conn(0x7fc0740cd140 msgr2=0x7fc07415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 --2- 172.21.15.123:0/3417368490 >> v2:172.21.15.146:3300/0 conn(0x7fc0740cd140 0x7fc07415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 >> v1:172.21.15.146:6789/0 conn(0x7fc074162540 legacy=0x7fc0740af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07406cce0 con 0x7fc07406daa0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fc064001670 con 0x7fc07406daa0 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 >> v1:172.21.15.123:6789/0 conn(0x7fc07406daa0 legacy=0x7fc0740cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0580039b0 0x7fc058003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 0x7fc058004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058005270 0x7fc05800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc07406deb0 con 0x7fc058004800 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc058010610 con 0x7fc058005270 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc058010810 con 0x7fc0580039b0 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.800+0000 7fc073e00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 0x7fc058004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc079200640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0580039b0 0x7fc058003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc073e00640 1 -- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0580039b0 msgr2=0x7fc058003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc073e00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0580039b0 0x7fc058003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc079c00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058005270 0x7fc05800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc073e00640 1 -- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058005270 msgr2=0x7fc05800fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc073e00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058005270 0x7fc05800fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc079c00640 1 -- 172.21.15.123:0/3417368490 reap_dead start 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc073e00640 1 -- 172.21.15.123:0/3417368490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc058003520 con 0x7fc058004800 2024-04-25T19:03:42.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.801+0000 7fc079200640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0580039b0 0x7fc058003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:42.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.802+0000 7fc073e00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 0x7fc058004c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00ba30 tx=0x7fc058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fc21229dd89387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.802+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c030070 con 0x7fc058004800 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.802+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05c02ade0 con 0x7fc058004800 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.802+0000 7fc073400640 1 -- 172.21.15.123:0/3417368490 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00b600 con 0x7fc058004800 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 -- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 msgr2=0x7fc058004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 0x7fc058004c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00ba30 tx=0x7fc058005830 comp rx=0 tx=0).stop 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 -- 172.21.15.123:0/3417368490 shutdown_connections 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 --2- 172.21.15.123:0/3417368490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc058004800 0x7fc058004c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 -- 172.21.15.123:0/3417368490 >> 172.21.15.123:0/3417368490 conn(0x7fc0740ce390 msgr2=0x7fc0740ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 -- 172.21.15.123:0/3417368490 shutdown_connections 2024-04-25T19:03:42.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.803+0000 7fc07bc00640 1 -- 172.21.15.123:0/3417368490 wait complete. 2024-04-25T19:03:42.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.805+0000 7fc07bc00640 1 Processor -- start 2024-04-25T19:03:42.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.805+0000 7fc07bc00640 1 -- start start 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 0x7fc0741607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc074160ce0 0x7fc07415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc07406d2a0 con 0x7fc07415acd0 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc07406d610 con 0x7fc058004800 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc07bc00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc074162950 con 0x7fc074160ce0 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52280/0 (socket says 172.21.15.123:52280) 2024-04-25T19:03:42.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 -- 172.21.15.123:0/1361387903 learned_addr learned my addr 172.21.15.123:0/1361387903 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079200640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 0x7fc0741607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc073e00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc074160ce0 0x7fc07415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 -- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc074160ce0 msgr2=0x7fc07415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc074160ce0 0x7fc07415a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 -- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 msgr2=0x7fc0741607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.807+0000 7fc079c00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 0x7fc0741607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc079c00640 1 -- 172.21.15.123:0/1361387903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07415b0e0 con 0x7fc07415acd0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc079200640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 0x7fc0741607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc079c00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0740ae890 tx=0x7fc06c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c018070 con 0x7fc07415acd0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c003b70 con 0x7fc07415acd0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c005070 con 0x7fc07415acd0 2024-04-25T19:03:42.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc074161dc0 con 0x7fc07415acd0 2024-04-25T19:03:42.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.808+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc074161800 con 0x7fc07415acd0 2024-04-25T19:03:42.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.809+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074076a40 con 0x7fc07415acd0 2024-04-25T19:03:42.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.811+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc06c01b070 con 0x7fc07415acd0 2024-04-25T19:03:42.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.812+0000 7fc070c00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 0x7fc0500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:42.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.812+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc06c094cb0 con 0x7fc07415acd0 2024-04-25T19:03:42.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.812+0000 7fc079200640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 0x7fc0500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:42.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.813+0000 7fc079200640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 0x7fc0500769f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0640017e0 tx=0x7fc064001470 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:42.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:42.817+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc07415b0e0 con 0x7fc07415acd0 2024-04-25T19:03:43.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.018+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc0740cfe10 con 0x7fc07415acd0 2024-04-25T19:03:43.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.018+0000 7fc070c00640 1 -- 172.21.15.123:0/1361387903 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fc06c0670b0 con 0x7fc07415acd0 2024-04-25T19:03:43.020 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:43.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 msgr2=0x7fc0500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 0x7fc0500769f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0640017e0 tx=0x7fc064001470 comp rx=0 tx=0).stop 2024-04-25T19:03:43.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 msgr2=0x7fc0740b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0740ae890 tx=0x7fc06c00dc00 comp rx=0 tx=0).stop 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 shutdown_connections 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc0500744e0 0x7fc0500769f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc07415acd0 0x7fc0740b44f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc074160ce0 0x7fc07415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 --2- 172.21.15.123:0/1361387903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc058004800 0x7fc0741607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.021+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 >> 172.21.15.123:0/1361387903 conn(0x7fc0740ce390 msgr2=0x7fc0740af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.022+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 shutdown_connections 2024-04-25T19:03:43.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.022+0000 7fc07bc00640 1 -- 172.21.15.123:0/1361387903 wait complete. 2024-04-25T19:03:43.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:43.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.314+0000 7f69a3800640 1 Processor -- start 2024-04-25T19:03:43.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.315+0000 7f69a3800640 1 -- start start 2024-04-25T19:03:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f69a3800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f69a3800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f699c161b10 0x7f699c161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f69a3800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f699c162460 con 0x7f699c154950 2024-04-25T19:03:43.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f69a3800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f699c0d1530 con 0x7f699c161b10 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f69a3800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f699c0aec30 con 0x7f699c0d1120 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f6993e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f699c161b10 0x7f699c161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f6993e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f699c161b10 0x7f699c161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52288/0 (socket says 172.21.15.123:52288) 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.317+0000 7f6993e00640 1 -- 172.21.15.123:0/2975933406 learned_addr learned my addr 172.21.15.123:0/2975933406 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.318+0000 7f69a0e00640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.318+0000 7f6993e00640 1 -- 172.21.15.123:0/2975933406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 msgr2=0x7f699c070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.318+0000 7f6993e00640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.318+0000 7f6993e00640 1 -- 172.21.15.123:0/2975933406 >> v1:172.21.15.146:6789/0 conn(0x7f699c154950 legacy=0x7f699c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.318+0000 7f6993e00640 1 -- 172.21.15.123:0/2975933406 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c0aef10 con 0x7f699c161b10 2024-04-25T19:03:43.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.319+0000 7f6993e00640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c161b10 0x7f699c161f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6984009870 tx=0x7f698402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b6584ed7ec80b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.319+0000 7f699be00640 1 -- 172.21.15.123:0/2975933406 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984037070 con 0x7f699c161b10 2024-04-25T19:03:43.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.319+0000 7f699be00640 1 -- 172.21.15.123:0/2975933406 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698402ae10 con 0x7f699c161b10 2024-04-25T19:03:43.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.319+0000 7f699be00640 1 -- 172.21.15.123:0/2975933406 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984032660 con 0x7f699c161b10 2024-04-25T19:03:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 -- 172.21.15.123:0/2975933406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c161b10 msgr2=0x7f699c161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c161b10 0x7f699c161f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6984009870 tx=0x7f698402a100 comp rx=0 tx=0).stop 2024-04-25T19:03:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 -- 172.21.15.123:0/2975933406 shutdown_connections 2024-04-25T19:03:43.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c161b10 0x7f699c161f20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 --2- 172.21.15.123:0/2975933406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 -- 172.21.15.123:0/2975933406 >> 172.21.15.123:0/2975933406 conn(0x7f699c0d1ae0 msgr2=0x7f699c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 -- 172.21.15.123:0/2975933406 shutdown_connections 2024-04-25T19:03:43.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.320+0000 7f69a3800640 1 -- 172.21.15.123:0/2975933406 wait complete. 2024-04-25T19:03:43.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.322+0000 7f69a3800640 1 Processor -- start 2024-04-25T19:03:43.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.322+0000 7f69a3800640 1 -- start start 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f699c161b10 0x7f699c142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f699c071430 con 0x7f699c154950 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f699c0af4d0 con 0x7f699c161b10 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a3800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f699c0715b0 con 0x7f699c0d1120 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52300/0 (socket says 172.21.15.123:52300) 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 -- 172.21.15.123:0/1680095375 learned_addr learned my addr 172.21.15.123:0/1680095375 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a0e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 -- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 msgr2=0x7f699c144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f69a1800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f699c161b10 0x7f699c142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.324+0000 7f6993e00640 1 -- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f699c161b10 msgr2=0x7f699c142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6993e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f699c161b10 0x7f699c142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6993e00640 1 -- 172.21.15.123:0/1680095375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c141ec0 con 0x7f699c154950 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6993e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69840090b0 tx=0x7f69840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984041070 con 0x7f699c154950 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6984032e20 con 0x7f699c154950 2024-04-25T19:03:43.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1421a0 con 0x7f699c154950 2024-04-25T19:03:43.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984037040 con 0x7f699c154950 2024-04-25T19:03:43.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.325+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c155350 con 0x7f699c154950 2024-04-25T19:03:43.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.326+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c1e9f10 con 0x7f699c154950 2024-04-25T19:03:43.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.327+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6984044070 con 0x7f699c154950 2024-04-25T19:03:43.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.328+0000 7f6999600640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 0x7f69780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.328+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f69840b60b0 con 0x7f699c154950 2024-04-25T19:03:43.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.329+0000 7f69a0e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 0x7f69780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.329+0000 7f69a0e00640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 0x7f69780769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f698c006fd0 tx=0x7f698c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.333+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f699c162460 con 0x7f699c154950 2024-04-25T19:03:43.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.517+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f699c0b0490 con 0x7f699c154950 2024-04-25T19:03:43.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.520+0000 7f6999600640 1 -- 172.21.15.123:0/1680095375 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f699c0b0490 con 0x7f699c154950 2024-04-25T19:03:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:43.523 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 msgr2=0x7f69780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 0x7f69780769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f698c006fd0 tx=0x7f698c008040 comp rx=0 tx=0).stop 2024-04-25T19:03:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 msgr2=0x7f699c143f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69840090b0 tx=0x7f69840046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 shutdown_connections 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69780744b0 0x7f69780769c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f699c161b10 0x7f699c142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699c154950 0x7f699c143f90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 --2- 172.21.15.123:0/1680095375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f699c0d1120 0x7f699c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 >> 172.21.15.123:0/1680095375 conn(0x7f699c0d1ae0 msgr2=0x7f699c15e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 shutdown_connections 2024-04-25T19:03:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.523+0000 7f69a3800640 1 -- 172.21.15.123:0/1680095375 wait complete. 2024-04-25T19:03:43.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:43.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:43.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.818+0000 7f86b7800640 1 Processor -- start 2024-04-25T19:03:43.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.819+0000 7f86b7800640 1 -- start start 2024-04-25T19:03:43.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.820+0000 7f86b7800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b00cd140 0x7f86b015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.820+0000 7f86b7800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f86b0162540 0x7f86b00af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.820+0000 7f86b7800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b00cc5f0 con 0x7f86b006daa0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.820+0000 7f86b7800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f86b006deb0 con 0x7f86b0162540 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.820+0000 7f86b7800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86b006d610 con 0x7f86b00cd140 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86b4e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f86b006daa0 0x7f86b00cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49872/0 (socket says 172.21.15.123:49872) 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86b4e00640 1 -- 172.21.15.123:0/1905940399 learned_addr learned my addr 172.21.15.123:0/1905940399 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275390010 0 0) 0x7f86b00cc5f0 con 0x7f86b006daa0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8694003520 con 0x7f86b006daa0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86a7e00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b00cd140 0x7f86b015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86b5800640 1 --2- 172.21.15.123:0/1905940399 >> v2:172.21.15.146:3300/0 conn(0x7f86b0162540 0x7f86b00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.821+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1761138204 0 0) 0x7f8694003520 con 0x7f86b006daa0 2024-04-25T19:03:43.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b00cd140 msgr2=0x7f86b015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b00cd140 0x7f86b015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 >> v2:172.21.15.146:3300/0 conn(0x7f86b0162540 msgr2=0x7f86b00af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 --2- 172.21.15.123:0/1905940399 >> v2:172.21.15.146:3300/0 conn(0x7f86b0162540 0x7f86b00af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b006ca00 con 0x7f86b006daa0 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f86a0001670 con 0x7f86b006daa0 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 >> v1:172.21.15.123:6789/0 conn(0x7f86b006daa0 legacy=0x7f86b00cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86940039b0 0x7f8694003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8694004800 0x7f8694004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694005150 0x7f869400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006deb0 con 0x7f8694004800 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006d610 con 0x7f8694005150 2024-04-25T19:03:43.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f869400bf80 con 0x7f86940039b0 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86a7e00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8694004800 0x7f8694004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86b5800640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694005150 0x7f869400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.822+0000 7f86a7e00640 1 -- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86940039b0 msgr2=0x7f8694003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.823+0000 7f86a7e00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86940039b0 0x7f8694003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.823+0000 7f86a7e00640 1 -- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694005150 msgr2=0x7f869400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.823+0000 7f86a7e00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694005150 0x7f869400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.823+0000 7f86a7e00640 1 -- 172.21.15.123:0/1905940399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8694003520 con 0x7f8694004800 2024-04-25T19:03:43.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.823+0000 7f86b5800640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694005150 0x7f869400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:43.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.824+0000 7f86a7e00640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8694004800 0x7f8694004c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f869800ba30 tx=0x7f8694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e6d6ebd2b85332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.824+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698030070 con 0x7f8694004800 2024-04-25T19:03:43.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.824+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869802ae10 con 0x7f8694004800 2024-04-25T19:03:43.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.824+0000 7f86afe00640 1 -- 172.21.15.123:0/1905940399 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869800b600 con 0x7f8694004800 2024-04-25T19:03:43.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 -- 172.21.15.123:0/1905940399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8694004800 msgr2=0x7f8694004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 --2- 172.21.15.123:0/1905940399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8694004800 0x7f8694004c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f869800ba30 tx=0x7f8694005830 comp rx=0 tx=0).stop 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b5800640 1 -- 172.21.15.123:0/1905940399 reap_dead start 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 -- 172.21.15.123:0/1905940399 shutdown_connections 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 -- 172.21.15.123:0/1905940399 >> 172.21.15.123:0/1905940399 conn(0x7f86b00ce390 msgr2=0x7f86b00ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 -- 172.21.15.123:0/1905940399 shutdown_connections 2024-04-25T19:03:43.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.825+0000 7f86b7800640 1 -- 172.21.15.123:0/1905940399 wait complete. 2024-04-25T19:03:43.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.827+0000 7f86b7800640 1 Processor -- start 2024-04-25T19:03:43.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.827+0000 7f86b7800640 1 -- start start 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 0x7f86b0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b014d1e0 0x7f86b009c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006d2a0 con 0x7f86b0160100 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006cfc0 con 0x7f8694004800 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b7800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86b0162950 con 0x7f86b014d1e0 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52330/0 (socket says 172.21.15.123:52330) 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 -- 172.21.15.123:0/4255642434 learned_addr learned my addr 172.21.15.123:0/4255642434 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b4e00640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 0x7f86b0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 -- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b014d1e0 msgr2=0x7f86b009c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b014d1e0 0x7f86b009c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 -- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 msgr2=0x7f86b0150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 0x7f86b0150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b5800640 1 -- 172.21.15.123:0/4255642434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b0162010 con 0x7f86b0160100 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.829+0000 7f86b4e00640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 0x7f86b0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86b5800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f86b0073240 tx=0x7f86a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8019070 con 0x7f86b0160100 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8007e00 con 0x7f86b0160100 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8014640 con 0x7f86b0160100 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b009eaa0 con 0x7f86b0160100 2024-04-25T19:03:43.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.830+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01622f0 con 0x7f86b0160100 2024-04-25T19:03:43.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.831+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b0076a40 con 0x7f86b0160100 2024-04-25T19:03:43.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.832+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f86a801c070 con 0x7f86b0160100 2024-04-25T19:03:43.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.833+0000 7f86ad600640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 0x7f868c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:43.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.833+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f86a8095180 con 0x7f86b0160100 2024-04-25T19:03:43.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.834+0000 7f86b4e00640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 0x7f868c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:43.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.834+0000 7f86b4e00640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 0x7f868c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86a0001eb0 tx=0x7f86a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:43.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:43.838+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86a805f630 con 0x7f86b0160100 2024-04-25T19:03:44.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.023+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86b0161d60 con 0x7f86b0160100 2024-04-25T19:03:44.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.023+0000 7f86ad600640 1 -- 172.21.15.123:0/4255642434 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f86a805ef30 con 0x7f86b0160100 2024-04-25T19:03:44.025 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 msgr2=0x7f868c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 0x7f868c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86a0001eb0 tx=0x7f86a0009040 comp rx=0 tx=0).stop 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 msgr2=0x7f86b01f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f86b0073240 tx=0x7f86a8009da0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 shutdown_connections 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f868c0744e0 0x7f868c0769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86b0160100 0x7f86b01f71e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86b014d1e0 0x7f86b009c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 --2- 172.21.15.123:0/4255642434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8694004800 0x7f86b0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 >> 172.21.15.123:0/4255642434 conn(0x7f86b00ce390 msgr2=0x7f86b00b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:44.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 shutdown_connections 2024-04-25T19:03:44.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.026+0000 7f86b7800640 1 -- 172.21.15.123:0/4255642434 wait complete. 2024-04-25T19:03:44.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:44.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.307+0000 7f6a9fa00640 1 Processor -- start 2024-04-25T19:03:44.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.307+0000 7f6a9fa00640 1 -- start start 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9fa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a981628f0 0x7f6a980b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9fa00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a9806e130 0x7f6a9815e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9fa00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a980b37b0 con 0x7f6a980b3a90 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9fa00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6a98161e10 con 0x7f6a981628f0 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9fa00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a981620f0 con 0x7f6a9806e130 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a981628f0 0x7f6a980b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a981628f0 0x7f6a980b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52346/0 (socket says 172.21.15.123:52346) 2024-04-25T19:03:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.309+0000 7f6a9d000640 1 -- 172.21.15.123:0/1535567798 learned_addr learned my addr 172.21.15.123:0/1535567798 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.310+0000 7f6a9d000640 1 -- 172.21.15.123:0/1535567798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a9806e130 msgr2=0x7f6a9815e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.310+0000 7f6a9d000640 1 --2- 172.21.15.123:0/1535567798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a9806e130 0x7f6a9815e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.310+0000 7f6a9d000640 1 -- 172.21.15.123:0/1535567798 >> v1:172.21.15.146:6789/0 conn(0x7f6a980b3a90 legacy=0x7f6a980b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.310+0000 7f6a9d000640 1 -- 172.21.15.123:0/1535567798 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a981623d0 con 0x7f6a981628f0 2024-04-25T19:03:44.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.311+0000 7f6a9d000640 1 --2- 172.21.15.123:0/1535567798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b31d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009870 tx=0x7f6a8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdad059cfe1bb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.311+0000 7f6a97400640 1 -- 172.21.15.123:0/1535567798 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037070 con 0x7f6a981628f0 2024-04-25T19:03:44.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.311+0000 7f6a97400640 1 -- 172.21.15.123:0/1535567798 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88032440 con 0x7f6a981628f0 2024-04-25T19:03:44.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.311+0000 7f6a97400640 1 -- 172.21.15.123:0/1535567798 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88032810 con 0x7f6a981628f0 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 -- 172.21.15.123:0/1535567798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 msgr2=0x7f6a980b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/1535567798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b31d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009870 tx=0x7f6a8802a210 comp rx=0 tx=0).stop 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 -- 172.21.15.123:0/1535567798 shutdown_connections 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/1535567798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a9806e130 0x7f6a9815e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/1535567798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b31d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 -- 172.21.15.123:0/1535567798 >> 172.21.15.123:0/1535567798 conn(0x7f6a9809b4a0 msgr2=0x7f6a980b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:44.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 -- 172.21.15.123:0/1535567798 shutdown_connections 2024-04-25T19:03:44.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.312+0000 7f6a9fa00640 1 -- 172.21.15.123:0/1535567798 wait complete. 2024-04-25T19:03:44.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.315+0000 7f6a9fa00640 1 Processor -- start 2024-04-25T19:03:44.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.315+0000 7f6a9fa00640 1 -- start start 2024-04-25T19:03:44.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 0x7f6a981f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 0x7f6a980afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980b2050 con 0x7f6a981628f0 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980b1ed0 con 0x7f6a9806e130 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.317+0000 7f6a9fa00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a980b21d0 con 0x7f6a980b3a90 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52358/0 (socket says 172.21.15.123:52358) 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 -- 172.21.15.123:0/3593972418 learned_addr learned my addr 172.21.15.123:0/3593972418 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9d000640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 0x7f6a981f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a97e00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 0x7f6a980afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 -- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 msgr2=0x7f6a980afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 0x7f6a980afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 -- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 msgr2=0x7f6a981f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 0x7f6a981f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 -- 172.21.15.123:0/3593972418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a980ca580 con 0x7f6a981628f0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9d000640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 0x7f6a981f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a97e00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 0x7f6a980afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a9da00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000b9e0 tx=0x7f6a9000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.318+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90019070 con 0x7f6a981628f0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.319+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9000cb20 con 0x7f6a981628f0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.319+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a980c6220 con 0x7f6a981628f0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.319+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90014440 con 0x7f6a981628f0 2024-04-25T19:03:44.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.319+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a980c67e0 con 0x7f6a981628f0 2024-04-25T19:03:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.320+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a98076a40 con 0x7f6a981628f0 2024-04-25T19:03:44.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.321+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a9001c070 con 0x7f6a981628f0 2024-04-25T19:03:44.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.322+0000 7f6a94c00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 0x7f6a740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.322+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f6a90095350 con 0x7f6a981628f0 2024-04-25T19:03:44.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.322+0000 7f6a9d000640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 0x7f6a740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.323+0000 7f6a9d000640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 0x7f6a740769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a88034000 tx=0x7f6a8802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.326+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a9005f760 con 0x7f6a981628f0 2024-04-25T19:03:44.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.510+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a9806d950 con 0x7f6a981628f0 2024-04-25T19:03:44.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.511+0000 7f6a94c00640 1 -- 172.21.15.123:0/3593972418 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f6a9005f060 con 0x7f6a981628f0 2024-04-25T19:03:44.513 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 msgr2=0x7f6a740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 0x7f6a740769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a88034000 tx=0x7f6a8802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 msgr2=0x7f6a980b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000b9e0 tx=0x7f6a9000bea0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 shutdown_connections 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6a740744b0 0x7f6a740769c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a981628f0 0x7f6a980b0430 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a980b3a90 0x7f6a980afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 --2- 172.21.15.123:0/3593972418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a9806e130 0x7f6a981f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 >> 172.21.15.123:0/3593972418 conn(0x7f6a9809b4a0 msgr2=0x7f6a98070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:44.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 shutdown_connections 2024-04-25T19:03:44.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.513+0000 7f6a9fa00640 1 -- 172.21.15.123:0/3593972418 wait complete. 2024-04-25T19:03:44.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:44.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.802+0000 7f14b1a00640 1 Processor -- start 2024-04-25T19:03:44.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.802+0000 7f14b1a00640 1 -- start start 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.804+0000 7f14b1a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 0x7f14ac161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.804+0000 7f14b1a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac162410 con 0x7f14ac162000 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.804+0000 7f14b1a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac0b4b40 con 0x7f14ac0b4730 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.804+0000 7f14b1a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac0d1780 con 0x7f14ac15f690 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.804+0000 7f14ab400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f14ac162000 0x7f14ac0b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49880/0 (socket says 172.21.15.123:49880) 2024-04-25T19:03:44.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14ab400640 1 -- 172.21.15.123:0/2969343497 learned_addr learned my addr 172.21.15.123:0/2969343497 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14abe00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 0x7f14ac161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532874444 0 0) 0x7f1494001110 con 0x7f14ac162000 2024-04-25T19:03:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f148c003520 con 0x7f14ac162000 2024-04-25T19:03:44.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3458626425 0 0) 0x7f148c003520 con 0x7f14ac162000 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 msgr2=0x7f14ac161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 0x7f14ac161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 >> v1:172.21.15.146:6789/0 conn(0x7f14ac0b4730 legacy=0x7f14ac15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac15faa0 con 0x7f14ac162000 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f1494001840 con 0x7f14ac162000 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 >> v1:172.21.15.123:6789/0 conn(0x7f14ac162000 legacy=0x7f14ac0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 0x7f148c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c003f60 0x7f148c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c004d40 0x7f148c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.805+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00bb50 con 0x7f148c004d40 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00bcd0 con 0x7f148c003f60 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f148c00bed0 con 0x7f14ac15f690 2024-04-25T19:03:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c004d40 0x7f148c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 -- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 msgr2=0x7f148c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14ab400640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c003f60 0x7f148c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac15f690 0x7f148c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 -- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c003f60 msgr2=0x7f148c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c003f60 0x7f148c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.806+0000 7f14b1000640 1 -- 172.21.15.123:0/2969343497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c003520 con 0x7f148c004d40 2024-04-25T19:03:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.807+0000 7f14b1000640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c004d40 0x7f148c005150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001e50 tx=0x7f148c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2144da6259a7ebc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.807+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4001a70 con 0x7f148c004d40 2024-04-25T19:03:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.807+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4003970 con 0x7f148c004d40 2024-04-25T19:03:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.807+0000 7f14aaa00640 1 -- 172.21.15.123:0/2969343497 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4003d60 con 0x7f148c004d40 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 -- 172.21.15.123:0/2969343497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c004d40 msgr2=0x7f148c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 --2- 172.21.15.123:0/2969343497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c004d40 0x7f148c005150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001e50 tx=0x7f148c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1000640 1 -- 172.21.15.123:0/2969343497 reap_dead start 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 -- 172.21.15.123:0/2969343497 shutdown_connections 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 -- 172.21.15.123:0/2969343497 >> 172.21.15.123:0/2969343497 conn(0x7f14ac09b4a0 msgr2=0x7f14ac09b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 -- 172.21.15.123:0/2969343497 shutdown_connections 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.808+0000 7f14b1a00640 1 -- 172.21.15.123:0/2969343497 wait complete. 2024-04-25T19:03:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.810+0000 7f14b1a00640 1 Processor -- start 2024-04-25T19:03:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.811+0000 7f14b1a00640 1 -- start start 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac14f6d0 0x7f14ac1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 0x7f14ac159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0d19b0 con 0x7f14ac0b35a0 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0b50b0 con 0x7f14ac159660 2024-04-25T19:03:44.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac0d1c90 con 0x7f14ac14f6d0 2024-04-25T19:03:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52384/0 (socket says 172.21.15.123:52384) 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.812+0000 7f14b1000640 1 -- 172.21.15.123:0/386412127 learned_addr learned my addr 172.21.15.123:0/386412127 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14ab400640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 0x7f14ac159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14abe00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac14f6d0 0x7f14ac1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 -- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac14f6d0 msgr2=0x7f14ac1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac14f6d0 0x7f14ac1519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 -- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 msgr2=0x7f14ac159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 0x7f14ac159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 -- 172.21.15.123:0/386412127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac159fb0 con 0x7f14ac0b35a0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14ab400640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 0x7f14ac159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1000640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001f70 tx=0x7f14a40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4041070 con 0x7f14ac0b35a0 2024-04-25T19:03:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4003d30 con 0x7f14ac0b35a0 2024-04-25T19:03:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.813+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac14fb30 con 0x7f14ac0b35a0 2024-04-25T19:03:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.814+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4001a70 con 0x7f14ac0b35a0 2024-04-25T19:03:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.814+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1526a0 con 0x7f14ac0b35a0 2024-04-25T19:03:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.815+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac153d10 con 0x7f14ac0b35a0 2024-04-25T19:03:44.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.816+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14a4044070 con 0x7f14ac0b35a0 2024-04-25T19:03:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.817+0000 7f14a3e00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 0x7f14880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.817+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f14a40b4a80 con 0x7f14ac0b35a0 2024-04-25T19:03:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.817+0000 7f14abe00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 0x7f14880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:44.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.818+0000 7f14abe00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 0x7f14880769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f149c004770 tx=0x7f149c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:44.821+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14ac159fb0 con 0x7f14ac0b35a0 2024-04-25T19:03:45.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.006+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14ac15eaf0 con 0x7f14ac0b35a0 2024-04-25T19:03:45.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.007+0000 7f14a3e00640 1 -- 172.21.15.123:0/386412127 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f14a407eea0 con 0x7f14ac0b35a0 2024-04-25T19:03:45.009 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 msgr2=0x7f14880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 0x7f14880769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f149c004770 tx=0x7f149c008040 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 msgr2=0x7f14ac0b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f14a4001f70 tx=0x7f14a40014a0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 shutdown_connections 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f14880744b0 0x7f14880769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac0b35a0 0x7f14ac0b39b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14ac159660 0x7f14ac159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 --2- 172.21.15.123:0/386412127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14ac14f6d0 0x7f14ac1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 >> 172.21.15.123:0/386412127 conn(0x7f14ac09b4a0 msgr2=0x7f14ac0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 shutdown_connections 2024-04-25T19:03:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.010+0000 7f14b1a00640 1 -- 172.21.15.123:0/386412127 wait complete. 2024-04-25T19:03:45.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:45.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.307+0000 7f03e5c00640 1 Processor -- start 2024-04-25T19:03:45.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.307+0000 7f03e5c00640 1 -- start start 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.309+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e015df10 0x7f03e00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.309+0000 7f03e5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03e00b0700 0x7f03e0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.309+0000 7f03e5c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e00d1430 con 0x7f03e006d580 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.309+0000 7f03e5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03e0070f40 con 0x7f03e00b0700 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.309+0000 7f03e5c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03e0161e90 con 0x7f03e015df10 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03d7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03e00b0700 0x7f03e0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03e5200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f03e006d580 0x7f03e006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33680/0 (socket says 172.21.15.123:33680) 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03e5200640 1 -- 172.21.15.123:0/3045736908 learned_addr learned my addr 172.21.15.123:0/3045736908 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:45.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03dfe00640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e015df10 0x7f03e00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03d7400640 1 -- 172.21.15.123:0/3045736908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e015df10 msgr2=0x7f03e00702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03d7400640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e015df10 0x7f03e00702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03d7400640 1 -- 172.21.15.123:0/3045736908 >> v1:172.21.15.123:6789/0 conn(0x7f03e006d580 legacy=0x7f03e006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03d7400640 1 -- 172.21.15.123:0/3045736908 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e0070980 con 0x7f03e00b0700 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.310+0000 7f03df400640 1 -- 172.21.15.123:0/3045736908 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618700296 0 0) 0x7f03e00d1430 con 0x7f03e006d580 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.311+0000 7f03d7400640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e00b0700 0x7f03e0162320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9aca5ddc4cbfbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03df400640 1 -- 172.21.15.123:0/3045736908 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8037070 con 0x7f03e00b0700 2024-04-25T19:03:45.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03df400640 1 -- 172.21.15.123:0/3045736908 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8032440 con 0x7f03e00b0700 2024-04-25T19:03:45.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03df400640 1 -- 172.21.15.123:0/3045736908 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8032810 con 0x7f03e00b0700 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 -- 172.21.15.123:0/3045736908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e00b0700 msgr2=0x7f03e0162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e00b0700 0x7f03e0162320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a180 comp rx=0 tx=0).stop 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 -- 172.21.15.123:0/3045736908 shutdown_connections 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e00b0700 0x7f03e0162320 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3045736908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e015df10 0x7f03e00702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.312+0000 7f03e5c00640 1 -- 172.21.15.123:0/3045736908 >> 172.21.15.123:0/3045736908 conn(0x7f03e0073090 msgr2=0x7f03e00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.313+0000 7f03e5c00640 1 -- 172.21.15.123:0/3045736908 shutdown_connections 2024-04-25T19:03:45.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.313+0000 7f03e5c00640 1 -- 172.21.15.123:0/3045736908 wait complete. 2024-04-25T19:03:45.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.315+0000 7f03e5c00640 1 Processor -- start 2024-04-25T19:03:45.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.315+0000 7f03e5c00640 1 -- start start 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.316+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 0x7f03e01f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.316+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e00b0700 0x7f03e01e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0162860 con 0x7f03e015df10 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0161b20 con 0x7f03e006d580 2024-04-25T19:03:45.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03e00cdfe0 con 0x7f03e00b0700 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56228/0 (socket says 172.21.15.123:56228) 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5200640 1 -- 172.21.15.123:0/3057648011 learned_addr learned my addr 172.21.15.123:0/3057648011 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03dfe00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 0x7f03e01f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5200640 1 -- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e00b0700 msgr2=0x7f03e01e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.317+0000 7f03e5200640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e00b0700 0x7f03e01e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5200640 1 -- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 msgr2=0x7f03e01f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5200640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 0x7f03e01f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5200640 1 -- 172.21.15.123:0/3057648011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e01f9440 con 0x7f03e015df10 2024-04-25T19:03:45.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03dfe00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 0x7f03e01f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:45.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5200640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c410 tx=0x7f03d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80074e0 con 0x7f03e015df10 2024-04-25T19:03:45.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d80047e0 con 0x7f03e015df10 2024-04-25T19:03:45.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01e82b0 con 0x7f03e015df10 2024-04-25T19:03:45.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8004bb0 con 0x7f03e015df10 2024-04-25T19:03:45.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.318+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01e8870 con 0x7f03e015df10 2024-04-25T19:03:45.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.319+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e0050740 con 0x7f03e015df10 2024-04-25T19:03:45.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.321+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f03d8011070 con 0x7f03e015df10 2024-04-25T19:03:45.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.322+0000 7f03dcc00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 0x7f03bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.322+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f03d80951b0 con 0x7f03e015df10 2024-04-25T19:03:45.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.322+0000 7f03dfe00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 0x7f03bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.323+0000 7f03dfe00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 0x7f03bc0769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03e01618e0 tx=0x7f03d000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.327+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03d805f530 con 0x7f03e015df10 2024-04-25T19:03:45.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.529+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f03e0074190 con 0x7f03e015df10 2024-04-25T19:03:45.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.530+0000 7f03dcc00640 1 -- 172.21.15.123:0/3057648011 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f03d805ee30 con 0x7f03e015df10 2024-04-25T19:03:45.532 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:45.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 msgr2=0x7f03bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 0x7f03bc0769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03e01618e0 tx=0x7f03d000b040 comp rx=0 tx=0).stop 2024-04-25T19:03:45.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 msgr2=0x7f03e00b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c410 tx=0x7f03d800c8d0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 shutdown_connections 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f03bc0744e0 0x7f03bc0769f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e015df10 0x7f03e00b2390 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03e00b0700 0x7f03e01e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 --2- 172.21.15.123:0/3057648011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e006d580 0x7f03e01f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 >> 172.21.15.123:0/3057648011 conn(0x7f03e0073090 msgr2=0x7f03e00ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 shutdown_connections 2024-04-25T19:03:45.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.533+0000 7f03e5c00640 1 -- 172.21.15.123:0/3057648011 wait complete. 2024-04-25T19:03:45.617 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-04-25T19:03:45.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:45.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.894+0000 7f0c53200640 1 Processor -- start 2024-04-25T19:03:45.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.894+0000 7f0c53200640 1 -- start start 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.896+0000 7f0c53200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 0x7f0c5406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.896+0000 7f0c53200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5406ded0 con 0x7f0c540b0cc0 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.896+0000 7f0c53200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c541456e0 con 0x7f0c5406dac0 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.896+0000 7f0c53200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c541462f0 con 0x7f0c541450d0 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.897+0000 7f0c52800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0c540b0cc0 0x7f0c54146ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33696/0 (socket says 172.21.15.123:33696) 2024-04-25T19:03:45.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.897+0000 7f0c52800640 1 -- 172.21.15.123:0/678953739 learned_addr learned my addr 172.21.15.123:0/678953739 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.897+0000 7f0c51e00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 0x7f0c5406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.897+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764411558 0 0) 0x7f0c4c001490 con 0x7f0c540b0cc0 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.897+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c38003520 con 0x7f0c540b0cc0 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3011131783 0 0) 0x7f0c38003520 con 0x7f0c540b0cc0 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 msgr2=0x7f0c5406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:45.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 0x7f0c5406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 >> v1:172.21.15.146:6789/0 conn(0x7f0c5406dac0 legacy=0x7f0c540b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c541459c0 con 0x7f0c540b0cc0 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f0c4c001bc0 con 0x7f0c540b0cc0 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 >> v1:172.21.15.123:6789/0 conn(0x7f0c540b0cc0 legacy=0x7f0c54146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c380039e0 0x7f0c38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c38004800 0x7f0c38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 0x7f0c38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541462f0 con 0x7f0c380039e0 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3800baf0 con 0x7f0c38004800 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3800bcf0 con 0x7f0c541450d0 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c380039e0 0x7f0c38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 -- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 msgr2=0x7f0c38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c541450d0 0x7f0c38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 -- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c38004800 msgr2=0x7f0c38004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c38004800 0x7f0c38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.898+0000 7f0c51e00640 1 -- 172.21.15.123:0/678953739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c38003520 con 0x7f0c380039e0 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c51e00640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c380039e0 0x7f0c38003e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009a90 tx=0x7f0c38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c9762c3bb74f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44036070 con 0x7f0c380039e0 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c44031400 con 0x7f0c380039e0 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c50a00640 1 -- 172.21.15.123:0/678953739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44009690 con 0x7f0c380039e0 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c53200640 1 -- 172.21.15.123:0/678953739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c380039e0 msgr2=0x7f0c38003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c53200640 1 --2- 172.21.15.123:0/678953739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c380039e0 0x7f0c38003e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009a90 tx=0x7f0c38005830 comp rx=0 tx=0).stop 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.900+0000 7f0c52800640 1 -- 172.21.15.123:0/678953739 reap_dead start 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.901+0000 7f0c53200640 1 -- 172.21.15.123:0/678953739 shutdown_connections 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.901+0000 7f0c53200640 1 -- 172.21.15.123:0/678953739 >> 172.21.15.123:0/678953739 conn(0x7f0c54073200 msgr2=0x7f0c54073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.901+0000 7f0c53200640 1 -- 172.21.15.123:0/678953739 shutdown_connections 2024-04-25T19:03:45.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.901+0000 7f0c53200640 1 -- 172.21.15.123:0/678953739 wait complete. 2024-04-25T19:03:45.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.903+0000 7f0c53200640 1 Processor -- start 2024-04-25T19:03:45.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.903+0000 7f0c53200640 1 -- start start 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 0x7f0c5415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 0x7f0c541616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54145f80 con 0x7f0c54161c00 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0c540b10d0 con 0x7f0c540cd830 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c53200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5406d580 con 0x7f0c380039e0 2024-04-25T19:03:45.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c52800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c52800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56254/0 (socket says 172.21.15.123:56254) 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.905+0000 7f0c52800640 1 -- 172.21.15.123:0/2435553531 learned_addr learned my addr 172.21.15.123:0/2435553531 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c51e00640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 0x7f0c5415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c51400640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 0x7f0c541616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 -- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 msgr2=0x7f0c5415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 0x7f0c5415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 -- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 msgr2=0x7f0c541616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 0x7f0c541616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 -- 172.21.15.123:0/2435553531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c54130b00 con 0x7f0c54161c00 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c51e00640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 0x7f0c5415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c51400640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 0x7f0c541616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c52800640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c001730 tx=0x7f0c4c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00a5b0 con 0x7f0c54161c00 2024-04-25T19:03:45.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00a750 con 0x7f0c54161c00 2024-04-25T19:03:45.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c54130de0 con 0x7f0c54161c00 2024-04-25T19:03:45.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00ab20 con 0x7f0c54161c00 2024-04-25T19:03:45.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.906+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541313a0 con 0x7f0c54161c00 2024-04-25T19:03:45.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.908+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c54050740 con 0x7f0c54161c00 2024-04-25T19:03:45.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.909+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c4c00acc0 con 0x7f0c54161c00 2024-04-25T19:03:45.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.910+0000 7f0c4a000640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 0x7f0c300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:45.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.910+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f0c4c095050 con 0x7f0c54161c00 2024-04-25T19:03:45.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.911+0000 7f0c51e00640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 0x7f0c300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:45.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.911+0000 7f0c51e00640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 0x7f0c300769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c44033000 tx=0x7f0c44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:45.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:45.914+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c5406ded0 con 0x7f0c54161c00 2024-04-25T19:03:46.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.114+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c5406e0b0 con 0x7f0c54161c00 2024-04-25T19:03:46.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.117+0000 7f0c4a000640 1 -- 172.21.15.123:0/2435553531 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f0c4c05f670 con 0x7f0c54161c00 2024-04-25T19:03:46.119 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:46.119 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 msgr2=0x7f0c300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 0x7f0c300769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c44033000 tx=0x7f0c44008030 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 msgr2=0x7f0c54132800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c001730 tx=0x7f0c4c00cf00 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 shutdown_connections 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0c300744b0 0x7f0c300769c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c54161c00 0x7f0c54132800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c540cd830 0x7f0c541616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 --2- 172.21.15.123:0/2435553531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c380039e0 0x7f0c5415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 >> 172.21.15.123:0/2435553531 conn(0x7f0c54073200 msgr2=0x7f0c54144610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:46.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 shutdown_connections 2024-04-25T19:03:46.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.120+0000 7f0c53200640 1 -- 172.21.15.123:0/2435553531 wait complete. 2024-04-25T19:03:46.151 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:46.152 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.423+0000 7f194e200640 1 Processor -- start 2024-04-25T19:03:46.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.423+0000 7f194e200640 1 -- start start 2024-04-25T19:03:46.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.425+0000 7f194e200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f194815df10 0x7f19480702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194e200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 0x7f1948162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194e200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 0x7f194806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194e200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1948070f40 con 0x7f194815df10 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194e200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1948161e90 con 0x7f194806d580 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194e200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19480d1430 con 0x7f19480b0700 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f194815df10 0x7f19480702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f194815df10 0x7f19480702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56268/0 (socket says 172.21.15.123:56268) 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194ce00640 1 -- 172.21.15.123:0/2273301154 learned_addr learned my addr 172.21.15.123:0/2273301154 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f1947e00640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 0x7f1948162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194ce00640 1 -- 172.21.15.123:0/2273301154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 msgr2=0x7f1948162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.426+0000 7f194d800640 1 --2- 172.21.15.123:0/2273301154 >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 0x7f194806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f194ce00640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 0x7f1948162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f194ce00640 1 -- 172.21.15.123:0/2273301154 >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 msgr2=0x7f194806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f194ce00640 1 --2- 172.21.15.123:0/2273301154 >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 0x7f194806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f194ce00640 1 -- 172.21.15.123:0/2273301154 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194806ded0 con 0x7f194815df10 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f1947e00640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 0x7f1948162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:46.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.427+0000 7f194d800640 1 --2- 172.21.15.123:0/2273301154 >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 0x7f194806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.428+0000 7f194ce00640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194815df10 0x7f19480702f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19480cde70 tx=0x7f193c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a548e66018d543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.428+0000 7f1947400640 1 -- 172.21.15.123:0/2273301154 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c037070 con 0x7f194815df10 2024-04-25T19:03:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.428+0000 7f1947400640 1 -- 172.21.15.123:0/2273301154 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c032440 con 0x7f194815df10 2024-04-25T19:03:46.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.428+0000 7f1947400640 1 -- 172.21.15.123:0/2273301154 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c032810 con 0x7f194815df10 2024-04-25T19:03:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 -- 172.21.15.123:0/2273301154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194815df10 msgr2=0x7f19480702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194815df10 0x7f19480702f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19480cde70 tx=0x7f193c02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 -- 172.21.15.123:0/2273301154 shutdown_connections 2024-04-25T19:03:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 --2- 172.21.15.123:0/2273301154 >> v2:172.21.15.146:3300/0 conn(0x7f194806d580 0x7f194806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19480b0700 0x7f1948162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 --2- 172.21.15.123:0/2273301154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194815df10 0x7f19480702f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 -- 172.21.15.123:0/2273301154 >> 172.21.15.123:0/2273301154 conn(0x7f1948073090 msgr2=0x7f19480734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 -- 172.21.15.123:0/2273301154 shutdown_connections 2024-04-25T19:03:46.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.429+0000 7f194e200640 1 -- 172.21.15.123:0/2273301154 wait complete. 2024-04-25T19:03:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.432+0000 7f194e200640 1 Processor -- start 2024-04-25T19:03:46.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.432+0000 7f194e200640 1 -- start start 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f194806d580 0x7f194814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f194815df10 0x7f19481586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1948161b20 con 0x7f19480b0700 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1948070c60 con 0x7f194815df10 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.433+0000 7f194e200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1948162860 con 0x7f194806d580 2024-04-25T19:03:46.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56284/0 (socket says 172.21.15.123:56284) 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 -- 172.21.15.123:0/3442099122 learned_addr learned my addr 172.21.15.123:0/3442099122 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f194ce00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f194806d580 0x7f194814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f194d800640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f194815df10 0x7f19481586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 -- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f194806d580 msgr2=0x7f194814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f194806d580 0x7f194814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 -- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f194815df10 msgr2=0x7f19481586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f194815df10 0x7f19481586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.434+0000 7f1947e00640 1 -- 172.21.15.123:0/3442099122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1948157600 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f1947e00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f193400c7f0 tx=0x7f193400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934015070 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1934007820 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934007bd0 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19481578e0 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.435+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1948157ea0 con 0x7f19480b0700 2024-04-25T19:03:46.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.436+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194815d6f0 con 0x7f19480b0700 2024-04-25T19:03:46.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.438+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1934018070 con 0x7f19480b0700 2024-04-25T19:03:46.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.439+0000 7f1944c00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 0x7f19280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.439+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1934095810 con 0x7f19480b0700 2024-04-25T19:03:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.439+0000 7f194ce00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 0x7f19280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.440+0000 7f194ce00640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 0x7f19280769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f193c034000 tx=0x7f193c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.443+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f193405fb90 con 0x7f19480b0700 2024-04-25T19:03:46.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.641+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1948050740 con 0x7f19480b0700 2024-04-25T19:03:46.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.643+0000 7f1944c00640 1 -- 172.21.15.123:0/3442099122 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f193405f490 con 0x7f19480b0700 2024-04-25T19:03:46.644 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:46.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 msgr2=0x7f19280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 0x7f19280769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f193c034000 tx=0x7f193c02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 msgr2=0x7f19481f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f193400c7f0 tx=0x7f193400ccb0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 shutdown_connections 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f19280744b0 0x7f19280769c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f194815df10 0x7f19481586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19480b0700 0x7f19481f5fa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 --2- 172.21.15.123:0/3442099122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f194806d580 0x7f194814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.645+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 >> 172.21.15.123:0/3442099122 conn(0x7f1948073090 msgr2=0x7f19480b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.646+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 shutdown_connections 2024-04-25T19:03:46.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.646+0000 7f194e200640 1 -- 172.21.15.123:0/3442099122 wait complete. 2024-04-25T19:03:46.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:46.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.943+0000 7fb801e00640 1 Processor -- start 2024-04-25T19:03:46.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.944+0000 7fb801e00640 1 -- start start 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb801e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7fc15dfc0 0x7fb7fc162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb801e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7fc15e3d0 con 0x7fb7fc06daa0 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb801e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7fc0cc5f0 con 0x7fb7fc0cd140 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb801e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb7fc06deb0 con 0x7fb7fc15dfc0 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb7faa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb7fc06daa0 0x7fb7fc0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33700/0 (socket says 172.21.15.123:33700) 2024-04-25T19:03:46.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.946+0000 7fb7faa00640 1 -- 172.21.15.123:0/1767707796 learned_addr learned my addr 172.21.15.123:0/1767707796 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020121649 0 0) 0x7fb7ec001110 con 0x7fb7fc06daa0 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e0003520 con 0x7fb7fc06daa0 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fb400640 1 --2- 172.21.15.123:0/1767707796 >> v2:172.21.15.146:3300/0 conn(0x7fb7fc15dfc0 0x7fb7fc162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3221877595 0 0) 0x7fb7e0003520 con 0x7fb7fc06daa0 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 >> v2:172.21.15.146:3300/0 conn(0x7fb7fc15dfc0 msgr2=0x7fb7fc162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 --2- 172.21.15.123:0/1767707796 >> v2:172.21.15.146:3300/0 conn(0x7fb7fc15dfc0 0x7fb7fc162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 >> v1:172.21.15.146:6789/0 conn(0x7fb7fc0cd140 legacy=0x7fb7fc0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc06ca00 con 0x7fb7fc06daa0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fb7ec001840 con 0x7fb7fc06daa0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 >> v1:172.21.15.123:6789/0 conn(0x7fb7fc06daa0 legacy=0x7fb7fc0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00039b0 0x7fb7e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 0x7fb7e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7e0005270 0x7fb7e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc06deb0 con 0x7fb7e0004800 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e000c1e0 con 0x7fb7e00039b0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.947+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e000c3e0 con 0x7fb7e0005270 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 0x7fb7e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7faa00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00039b0 0x7fb7e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7fb400640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7e0005270 0x7fb7e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 -- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7e0005270 msgr2=0x7fb7e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7e0005270 0x7fb7e000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 -- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00039b0 msgr2=0x7fb7e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:46.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00039b0 0x7fb7e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7f3e00640 1 -- 172.21.15.123:0/1767707796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e0003520 con 0x7fb7e0004800 2024-04-25T19:03:46.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.948+0000 7fb7fb400640 1 -- 172.21.15.123:0/1767707796 reap_dead start 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.949+0000 7fb7f3e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 0x7fb7e0004c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4003db0 tx=0x7fb7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916c805f0f10192c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.949+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037070 con 0x7fb7e0004800 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.949+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e402a9f0 con 0x7fb7e0004800 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.949+0000 7fb7fa000640 1 -- 172.21.15.123:0/1767707796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4003980 con 0x7fb7e0004800 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 -- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 msgr2=0x7fb7e0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 0x7fb7e0004c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4003db0 tx=0x7fb7e0005830 comp rx=0 tx=0).stop 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 -- 172.21.15.123:0/1767707796 shutdown_connections 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 --2- 172.21.15.123:0/1767707796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e0004800 0x7fb7e0004c10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 -- 172.21.15.123:0/1767707796 >> 172.21.15.123:0/1767707796 conn(0x7fb7fc0ce390 msgr2=0x7fb7fc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:46.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 -- 172.21.15.123:0/1767707796 shutdown_connections 2024-04-25T19:03:46.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.950+0000 7fb801e00640 1 -- 172.21.15.123:0/1767707796 wait complete. 2024-04-25T19:03:46.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.953+0000 7fb801e00640 1 Processor -- start 2024-04-25T19:03:46.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.953+0000 7fb801e00640 1 -- start start 2024-04-25T19:03:46.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e0004800 0x7fb7fc15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7fc157cc0 0x7fb7fc157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc06cfc0 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc06d610 con 0x7fb7e0004800 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.954+0000 7fb801e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7fc06d2a0 con 0x7fb7fc157cc0 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56306/0 (socket says 172.21.15.123:56306) 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 -- 172.21.15.123:0/3367540133 learned_addr learned my addr 172.21.15.123:0/3367540133 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7f3e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7fc157cc0 0x7fb7fc157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7faa00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e0004800 0x7fb7fc15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 -- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7fc157cc0 msgr2=0x7fb7fc157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7fc157cc0 0x7fb7fc157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 -- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e0004800 msgr2=0x7fb7fc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e0004800 0x7fb7fc15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 -- 172.21.15.123:0/3367540133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc15ab00 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.955+0000 7fb7fb400640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0ce180 tx=0x7fb7f400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.956+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007e20 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.956+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f401a070 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.956+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014610 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.956+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc157a70 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.956+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc159800 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.957+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7fc076a40 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.958+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7f40147b0 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.959+0000 7fb7f3400640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 0x7fb7d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.959+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fb7f4095b30 con 0x7fb7fc1f5ff0 2024-04-25T19:03:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.959+0000 7fb7faa00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 0x7fb7d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.960+0000 7fb7faa00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 0x7fb7d80769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:46.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:46.964+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7f405ff40 con 0x7fb7fc1f5ff0 2024-04-25T19:03:47.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.166+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7fc1592b0 con 0x7fb7fc1f5ff0 2024-04-25T19:03:47.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.167+0000 7fb7f3400640 1 -- 172.21.15.123:0/3367540133 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fb7f405f840 con 0x7fb7fc1f5ff0 2024-04-25T19:03:47.169 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 msgr2=0x7fb7d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 0x7fb7d80769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec000ed0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 msgr2=0x7fb7fc1f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0ce180 tx=0x7fb7f400f520 comp rx=0 tx=0).stop 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 shutdown_connections 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb7d80744b0 0x7fb7d80769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7fc1f5ff0 0x7fb7fc1f6400 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7fc157cc0 0x7fb7fc157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 --2- 172.21.15.123:0/3367540133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e0004800 0x7fb7fc15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 >> 172.21.15.123:0/3367540133 conn(0x7fb7fc0ce390 msgr2=0x7fb7fc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:47.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 shutdown_connections 2024-04-25T19:03:47.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.170+0000 7fb801e00640 1 -- 172.21.15.123:0/3367540133 wait complete. 2024-04-25T19:03:47.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.468+0000 7fe20f200640 1 Processor -- start 2024-04-25T19:03:47.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.468+0000 7fe20f200640 1 -- start start 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20f200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe210160350 0x7fe2100d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100b9800 con 0x7fe2100ce350 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20f200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100b3090 con 0x7fe2100b8130 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20f200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe2100d1e10 con 0x7fe210160350 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20d400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe2100ce350 0x7fe2100ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33702/0 (socket says 172.21.15.123:33702) 2024-04-25T19:03:47.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.470+0000 7fe20d400640 1 -- 172.21.15.123:0/118734265 learned_addr learned my addr 172.21.15.123:0/118734265 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635838771 0 0) 0x7fe2100b9800 con 0x7fe2100ce350 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1f4003520 con 0x7fe2100ce350 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20de00640 1 --2- 172.21.15.123:0/118734265 >> v2:172.21.15.146:3300/0 conn(0x7fe210160350 0x7fe2100d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 171210197 0 0) 0x7fe1f4003520 con 0x7fe2100ce350 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 >> v2:172.21.15.146:3300/0 conn(0x7fe210160350 msgr2=0x7fe2100d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 --2- 172.21.15.123:0/118734265 >> v2:172.21.15.146:3300/0 conn(0x7fe210160350 0x7fe2100d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 >> v1:172.21.15.146:6789/0 conn(0x7fe2100b8130 legacy=0x7fe2100b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2100b6790 con 0x7fe2100ce350 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fe1fc001670 con 0x7fe2100ce350 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 >> v1:172.21.15.123:6789/0 conn(0x7fe2100ce350 legacy=0x7fe2100ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1f40039b0 0x7fe1f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 0x7fe1f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4005350 0x7fe1f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100d1e10 con 0x7fe1f4004800 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f400c280 con 0x7fe1f40039b0 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f400c480 con 0x7fe1f4005350 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.471+0000 7fe20d400640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 0x7fe1f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20de00640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1f40039b0 0x7fe1f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20d400640 1 -- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4005350 msgr2=0x7fe1f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20d400640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4005350 0x7fe1f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20d400640 1 -- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1f40039b0 msgr2=0x7fe1f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20d400640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1f40039b0 0x7fe1f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20d400640 1 -- 172.21.15.123:0/118734265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f4003520 con 0x7fe1f4004800 2024-04-25T19:03:47.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20de00640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1f40039b0 0x7fe1f4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:47.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.472+0000 7fe20e800640 1 -- 172.21.15.123:0/118734265 reap_dead start 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.473+0000 7fe20d400640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 0x7fe1f4004c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c240 tx=0x7fe1f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f10dfb87d87394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.473+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc007d30 con 0x7fe1f4004800 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.473+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1fc005af0 con 0x7fe1f4004800 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.473+0000 7fe20ca00640 1 -- 172.21.15.123:0/118734265 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc007070 con 0x7fe1f4004800 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 -- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 msgr2=0x7fe1f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 0x7fe1f4004c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c240 tx=0x7fe1f4005830 comp rx=0 tx=0).stop 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 -- 172.21.15.123:0/118734265 shutdown_connections 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 --2- 172.21.15.123:0/118734265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f4004800 0x7fe1f4004c10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 -- 172.21.15.123:0/118734265 >> 172.21.15.123:0/118734265 conn(0x7fe2100d3830 msgr2=0x7fe2100d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 -- 172.21.15.123:0/118734265 shutdown_connections 2024-04-25T19:03:47.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.474+0000 7fe20f200640 1 -- 172.21.15.123:0/118734265 wait complete. 2024-04-25T19:03:47.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.476+0000 7fe20f200640 1 Processor -- start 2024-04-25T19:03:47.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.477+0000 7fe20f200640 1 -- start start 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4004800 0x7fe2100cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2101f99f0 0x7fe2100c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe210160760 con 0x7fe2100ccca0 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100d20f0 con 0x7fe2101f99f0 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2100ce760 con 0x7fe1f4004800 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.478+0000 7fe20d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56326/0 (socket says 172.21.15.123:56326) 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 -- 172.21.15.123:0/2254223926 learned_addr learned my addr 172.21.15.123:0/2254223926 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20de00640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4004800 0x7fe2100cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20e800640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2101f99f0 0x7fe2100c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 -- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4004800 msgr2=0x7fe2100cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4004800 0x7fe2100cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 -- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2101f99f0 msgr2=0x7fe2100c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2101f99f0 0x7fe2100c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 -- 172.21.15.123:0/2254223926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2101f9e00 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.479+0000 7fe20d400640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc001970 tx=0x7fe2100b6960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.480+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc041070 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.480+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1fc004e20 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.480+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2100c7590 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.480+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc007d30 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.480+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2100c7b50 con 0x7fe2100ccca0 2024-04-25T19:03:47.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.481+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2100c7300 con 0x7fe2100ccca0 2024-04-25T19:03:47.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.482+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1fc044070 con 0x7fe2100ccca0 2024-04-25T19:03:47.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.484+0000 7fe206000640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 0x7fe1f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.484+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fe1fc0b5f90 con 0x7fe2100ccca0 2024-04-25T19:03:47.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.484+0000 7fe20de00640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 0x7fe1f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.485+0000 7fe20de00640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 0x7fe1f00769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2100b2b30 tx=0x7fe20000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:47.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.488+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2101f9e00 con 0x7fe2100ccca0 2024-04-25T19:03:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.686+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe210050740 con 0x7fe2100ccca0 2024-04-25T19:03:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.687+0000 7fe206000640 1 -- 172.21.15.123:0/2254223926 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fe1fc080390 con 0x7fe2100ccca0 2024-04-25T19:03:47.689 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 msgr2=0x7fe1f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 0x7fe1f00769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2100b2b30 tx=0x7fe20000b040 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 msgr2=0x7fe2101f94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc001970 tx=0x7fe2100b6960 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 shutdown_connections 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe1f00744b0 0x7fe1f00769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2101f99f0 0x7fe2100c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2100ccca0 0x7fe2101f94b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 --2- 172.21.15.123:0/2254223926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1f4004800 0x7fe2100cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 >> 172.21.15.123:0/2254223926 conn(0x7fe2100d3830 msgr2=0x7fe2100b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 shutdown_connections 2024-04-25T19:03:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.690+0000 7fe20f200640 1 -- 172.21.15.123:0/2254223926 wait complete. 2024-04-25T19:03:47.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:47.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.987+0000 7fa434e00640 1 Processor -- start 2024-04-25T19:03:47.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.988+0000 7fa434e00640 1 -- start start 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa434e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa43015dfc0 0x7fa430162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa434e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43015e3d0 con 0x7fa4300cd140 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa434e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4300cc5f0 con 0x7fa43006daa0 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa434e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa43006deb0 con 0x7fa43015dfc0 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa425400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa4300cd140 0x7fa4300af040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33718/0 (socket says 172.21.15.123:33718) 2024-04-25T19:03:47.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.990+0000 7fa425400640 1 -- 172.21.15.123:0/2695123579 learned_addr learned my addr 172.21.15.123:0/2695123579 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694912163 0 0) 0x7fa418001110 con 0x7fa4300cd140 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa414003520 con 0x7fa4300cd140 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42e800640 1 --2- 172.21.15.123:0/2695123579 >> v2:172.21.15.146:3300/0 conn(0x7fa43015dfc0 0x7fa430162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1750608028 0 0) 0x7fa414003520 con 0x7fa4300cd140 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 >> v2:172.21.15.146:3300/0 conn(0x7fa43015dfc0 msgr2=0x7fa430162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 --2- 172.21.15.123:0/2695123579 >> v2:172.21.15.146:3300/0 conn(0x7fa43015dfc0 0x7fa430162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 >> v1:172.21.15.146:6789/0 conn(0x7fa43006daa0 legacy=0x7fa4300cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43006ca00 con 0x7fa4300cd140 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fa418001840 con 0x7fa4300cd140 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 >> v1:172.21.15.123:6789/0 conn(0x7fa4300cd140 legacy=0x7fa4300af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.991+0000 7fa42d400640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 0x7fa414003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42d400640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa414004800 0x7fa414004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42d400640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa43006deb0 con 0x7fa4140039b0 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa41400c160 con 0x7fa4140051c0 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa41400c360 con 0x7fa414004800 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 0x7fa414003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42e800640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 -- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa414004800 msgr2=0x7fa414004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa414004800 0x7fa414004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 -- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140051c0 msgr2=0x7fa41400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42de00640 1 -- 172.21.15.123:0/2695123579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa414003520 con 0x7fa4140039b0 2024-04-25T19:03:47.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.992+0000 7fa42e800640 1 -- 172.21.15.123:0/2695123579 reap_dead start 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.993+0000 7fa42de00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 0x7fa414003e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa420001720 tx=0x7fa414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b59958f1fa3d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.993+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420001370 con 0x7fa4140039b0 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.993+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4200052d0 con 0x7fa4140039b0 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.993+0000 7fa42d400640 1 -- 172.21.15.123:0/2695123579 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42002aaf0 con 0x7fa4140039b0 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 -- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 msgr2=0x7fa414003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 0x7fa414003e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa420001720 tx=0x7fa414005830 comp rx=0 tx=0).stop 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 -- 172.21.15.123:0/2695123579 shutdown_connections 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 --2- 172.21.15.123:0/2695123579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4140039b0 0x7fa414003e20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 -- 172.21.15.123:0/2695123579 >> 172.21.15.123:0/2695123579 conn(0x7fa4300ce390 msgr2=0x7fa4300ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 -- 172.21.15.123:0/2695123579 shutdown_connections 2024-04-25T19:03:47.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.994+0000 7fa434e00640 1 -- 172.21.15.123:0/2695123579 wait complete. 2024-04-25T19:03:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.997+0000 7fa434e00640 1 Processor -- start 2024-04-25T19:03:47.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.997+0000 7fa434e00640 1 -- start start 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140039b0 0x7fa430151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 0x7fa4300b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa43006cfc0 con 0x7fa43015feb0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa43006d610 con 0x7fa4140039b0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.998+0000 7fa434e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa43006d2a0 con 0x7fa4300b47b0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56340/0 (socket says 172.21.15.123:56340) 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 -- 172.21.15.123:0/4148178754 learned_addr learned my addr 172.21.15.123:0/4148178754 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42de00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140039b0 0x7fa430151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa425400640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 0x7fa4300b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 -- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 msgr2=0x7fa4300b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 0x7fa4300b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 -- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140039b0 msgr2=0x7fa430151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140039b0 0x7fa430151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa42e800640 1 -- 172.21.15.123:0/4148178754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa430160800 con 0x7fa43015feb0 2024-04-25T19:03:48.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:47.999+0000 7fa425400640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 0x7fa4300b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa42e800640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4300ce180 tx=0x7fa42800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428016420 con 0x7fa43015feb0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42801b070 con 0x7fa43015feb0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428016800 con 0x7fa43015feb0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4301515c0 con 0x7fa43015feb0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.000+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4300b5850 con 0x7fa43015feb0 2024-04-25T19:03:48.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.001+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4300b52e0 con 0x7fa43015feb0 2024-04-25T19:03:48.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.002+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4280169a0 con 0x7fa43015feb0 2024-04-25T19:03:48.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.003+0000 7fa426a00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 0x7fa40c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.003+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa428096390 con 0x7fa43015feb0 2024-04-25T19:03:48.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.004+0000 7fa42de00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 0x7fa40c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.004+0000 7fa42de00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 0x7fa40c0769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa420032000 tx=0x7fa42002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.017+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa430160800 con 0x7fa43015feb0 2024-04-25T19:03:48.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.210+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa430069c60 con 0x7fa43015feb0 2024-04-25T19:03:48.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.212+0000 7fa426a00640 1 -- 172.21.15.123:0/4148178754 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fa4280607b0 con 0x7fa43015feb0 2024-04-25T19:03:48.214 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 msgr2=0x7fa40c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 0x7fa40c0769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa420032000 tx=0x7fa42002e000 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 msgr2=0x7fa4301602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4300ce180 tx=0x7fa42800f520 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 shutdown_connections 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa40c0744e0 0x7fa40c0769f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43015feb0 0x7fa4301602c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4300b47b0 0x7fa4300b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 --2- 172.21.15.123:0/4148178754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140039b0 0x7fa430151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.214+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 >> 172.21.15.123:0/4148178754 conn(0x7fa4300ce390 msgr2=0x7fa43006f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.215+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 shutdown_connections 2024-04-25T19:03:48.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.215+0000 7fa434e00640 1 -- 172.21.15.123:0/4148178754 wait complete. 2024-04-25T19:03:48.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-25T19:03:48.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.508+0000 7fa0f2e00640 1 Processor -- start 2024-04-25T19:03:48.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.508+0000 7fa0f2e00640 1 -- start start 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0ec0d1120 0x7fa0ec070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec161b10 0x7fa0ec161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 0x7fa0ec154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0d1530 con 0x7fa0ec0d1120 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec162460 con 0x7fa0ec154950 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0f2e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec0aec30 con 0x7fa0ec161b10 2024-04-25T19:03:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.510+0000 7fa0ebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0ec0d1120 0x7fa0ec070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0ec0d1120 0x7fa0ec070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56348/0 (socket says 172.21.15.123:56348) 2024-04-25T19:03:48.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 -- 172.21.15.123:0/646998526 learned_addr learned my addr 172.21.15.123:0/646998526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:48.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0f0e00640 1 --2- 172.21.15.123:0/646998526 >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 0x7fa0ec154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0eb400640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec161b10 0x7fa0ec161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 -- 172.21.15.123:0/646998526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec161b10 msgr2=0x7fa0ec161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec161b10 0x7fa0ec161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 -- 172.21.15.123:0/646998526 >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 msgr2=0x7fa0ec154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/646998526 >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 0x7fa0ec154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0ebe00640 1 -- 172.21.15.123:0/646998526 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0aef10 con 0x7fa0ec0d1120 2024-04-25T19:03:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.511+0000 7fa0f0e00640 1 --2- 172.21.15.123:0/646998526 >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 0x7fa0ec154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.513+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec0d1120 0x7fa0ec070ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec070c90 tx=0x7fa0d402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4f55d17ddc749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.513+0000 7fa0eaa00640 1 -- 172.21.15.123:0/646998526 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4037070 con 0x7fa0ec0d1120 2024-04-25T19:03:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.513+0000 7fa0eaa00640 1 -- 172.21.15.123:0/646998526 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d402ae00 con 0x7fa0ec0d1120 2024-04-25T19:03:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.513+0000 7fa0eaa00640 1 -- 172.21.15.123:0/646998526 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4032660 con 0x7fa0ec0d1120 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 -- 172.21.15.123:0/646998526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec0d1120 msgr2=0x7fa0ec070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec0d1120 0x7fa0ec070ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec070c90 tx=0x7fa0d402a0b0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 -- 172.21.15.123:0/646998526 shutdown_connections 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/646998526 >> v2:172.21.15.146:3300/0 conn(0x7fa0ec154950 0x7fa0ec154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec161b10 0x7fa0ec161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/646998526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec0d1120 0x7fa0ec070ef0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 -- 172.21.15.123:0/646998526 >> 172.21.15.123:0/646998526 conn(0x7fa0ec0d1ae0 msgr2=0x7fa0ec0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 -- 172.21.15.123:0/646998526 shutdown_connections 2024-04-25T19:03:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.514+0000 7fa0f2e00640 1 -- 172.21.15.123:0/646998526 wait complete. 2024-04-25T19:03:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.516+0000 7fa0f2e00640 1 Processor -- start 2024-04-25T19:03:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.516+0000 7fa0f2e00640 1 -- start start 2024-04-25T19:03:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec0d1120 0x7fa0ec154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 0x7fa0ec159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1552a0 con 0x7fa0ec154950 2024-04-25T19:03:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0af4d0 con 0x7fa0ec161b10 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0f2e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec1f65e0 con 0x7fa0ec0d1120 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56356/0 (socket says 172.21.15.123:56356) 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.518+0000 7fa0eb400640 1 -- 172.21.15.123:0/2452186587 learned_addr learned my addr 172.21.15.123:0/2452186587 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec0d1120 0x7fa0ec154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0f0e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 0x7fa0ec159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 -- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec0d1120 msgr2=0x7fa0ec154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec0d1120 0x7fa0ec154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 -- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 msgr2=0x7fa0ec159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 0x7fa0ec159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 -- 172.21.15.123:0/2452186587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec158cf0 con 0x7fa0ec154950 2024-04-25T19:03:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0f0e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 0x7fa0ec159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0eb400640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc007e70 tx=0x7fa0dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc014620 con 0x7fa0ec154950 2024-04-25T19:03:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0dc019070 con 0x7fa0ec154950 2024-04-25T19:03:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc014a00 con 0x7fa0ec154950 2024-04-25T19:03:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec158fd0 con 0x7fa0ec154950 2024-04-25T19:03:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.519+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec159590 con 0x7fa0ec154950 2024-04-25T19:03:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.520+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec076a40 con 0x7fa0ec154950 2024-04-25T19:03:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.522+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0dc014ba0 con 0x7fa0ec154950 2024-04-25T19:03:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.523+0000 7fa0e3e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 0x7fa0c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.523+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa0dc095e60 con 0x7fa0ec154950 2024-04-25T19:03:48.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.523+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 0x7fa0c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.524+0000 7fa0ebe00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 0x7fa0c80769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4034000 tx=0x7fa0d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.528+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0dc060270 con 0x7fa0ec154950 2024-04-25T19:03:48.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:48.806+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fa0ec070250 con 0x7fa0ec154950 2024-04-25T19:03:49.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.148+0000 7fa0e3e00640 1 -- 172.21.15.123:0/2452186587 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v91) ==== 123+0+0 (secure 0 0 0) 0x7fa0dc05fb70 con 0x7fa0ec154950 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 msgr2=0x7fa0c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 0x7fa0c80769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4034000 tx=0x7fa0d402e000 comp rx=0 tx=0).stop 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 msgr2=0x7fa0ec1e9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc007e70 tx=0x7fa0dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 shutdown_connections 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa0c80744b0 0x7fa0c80769c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0ec161b10 0x7fa0ec159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0ec154950 0x7fa0ec1e9930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 --2- 172.21.15.123:0/2452186587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0ec0d1120 0x7fa0ec154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 >> 172.21.15.123:0/2452186587 conn(0x7fa0ec0d1ae0 msgr2=0x7fa0ec15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 shutdown_connections 2024-04-25T19:03:49.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.152+0000 7fa0f2e00640 1 -- 172.21.15.123:0/2452186587 wait complete. 2024-04-25T19:03:49.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T19:03:49.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.463+0000 7f073e800640 1 Processor -- start 2024-04-25T19:03:49.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.463+0000 7f073e800640 1 -- start start 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f073806dc90 0x7f07380702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073e800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07380d1430 con 0x7f073815df10 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073e800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0738070980 con 0x7f07380b0700 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073e800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0738070f40 con 0x7f073806dc90 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073d400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f073815df10 0x7f0738162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33724/0 (socket says 172.21.15.123:33724) 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073d400640 1 -- 172.21.15.123:0/1199841704 learned_addr learned my addr 172.21.15.123:0/1199841704 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:49.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.465+0000 7f073de00640 1 --2- 172.21.15.123:0/1199841704 >> v2:172.21.15.123:3300/0 conn(0x7f073806dc90 0x7f07380702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:49.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.466+0000 7f0737e00640 1 -- 172.21.15.123:0/1199841704 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972358416 0 0) 0x7f07380d1430 con 0x7f073815df10 2024-04-25T19:03:49.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.466+0000 7f0737e00640 1 -- 172.21.15.123:0/1199841704 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0720003520 con 0x7f073815df10 2024-04-25T19:03:49.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.466+0000 7f073de00640 1 -- 172.21.15.123:0/1199841704 >> v1:172.21.15.146:6789/0 conn(0x7f07380b0700 legacy=0x7f073806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:49.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.466+0000 7f073de00640 1 -- 172.21.15.123:0/1199841704 >> v1:172.21.15.123:6789/0 conn(0x7f073815df10 legacy=0x7f0738162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.466+0000 7f073de00640 1 -- 172.21.15.123:0/1199841704 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0738070c60 con 0x7f073806dc90 2024-04-25T19:03:49.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.467+0000 7f073de00640 1 --2- 172.21.15.123:0/1199841704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073806dc90 0x7f07380702f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f073002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b64c6a2dcf60f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:49.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.467+0000 7f0737e00640 1 -- 172.21.15.123:0/1199841704 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730007740 con 0x7f073806dc90 2024-04-25T19:03:49.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.467+0000 7f0737e00640 1 -- 172.21.15.123:0/1199841704 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07300078e0 con 0x7f073806dc90 2024-04-25T19:03:49.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.467+0000 7f0737e00640 1 -- 172.21.15.123:0/1199841704 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730007cb0 con 0x7f073806dc90 2024-04-25T19:03:49.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 -- 172.21.15.123:0/1199841704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073806dc90 msgr2=0x7f07380702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 --2- 172.21.15.123:0/1199841704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073806dc90 0x7f07380702f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f073002a6a0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 -- 172.21.15.123:0/1199841704 shutdown_connections 2024-04-25T19:03:49.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 --2- 172.21.15.123:0/1199841704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073806dc90 0x7f07380702f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 -- 172.21.15.123:0/1199841704 >> 172.21.15.123:0/1199841704 conn(0x7f0738073090 msgr2=0x7f07380734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:49.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 -- 172.21.15.123:0/1199841704 shutdown_connections 2024-04-25T19:03:49.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.468+0000 7f073e800640 1 -- 172.21.15.123:0/1199841704 wait complete. 2024-04-25T19:03:49.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.470+0000 7f073e800640 1 Processor -- start 2024-04-25T19:03:49.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.471+0000 7f073e800640 1 -- start start 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07380b0700 0x7f0738154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 0x7f0738157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07380cdfe0 con 0x7f073815df10 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0738161e90 con 0x7f0738156d20 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073e800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f07380ce160 con 0x7f07380b0700 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56366/0 (socket says 172.21.15.123:56366) 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.472+0000 7f073ca00640 1 -- 172.21.15.123:0/3293187194 learned_addr learned my addr 172.21.15.123:0/3293187194 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073de00640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 0x7f0738157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073d400640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07380b0700 0x7f0738154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 -- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07380b0700 msgr2=0x7f0738154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07380b0700 0x7f0738154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 -- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 msgr2=0x7f0738157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 0x7f0738157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 -- 172.21.15.123:0/3293187194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0738157670 con 0x7f073815df10 2024-04-25T19:03:49.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073de00640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 0x7f0738157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:49.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f073ca00640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f072800b3f0 tx=0x7f072800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.473+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728015020 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.474+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728011830 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.474+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728011c20 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.474+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0738159a40 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.474+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381f6aa0 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.475+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0738050740 con 0x7f073815df10 2024-04-25T19:03:49.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.476+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0728018020 con 0x7f073815df10 2024-04-25T19:03:49.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.478+0000 7f0735600640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 0x7f071c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:49.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.478+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f07280955c0 con 0x7f073815df10 2024-04-25T19:03:49.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.479+0000 7f073d400640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 0x7f071c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:49.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.479+0000 7f073d400640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 0x7f071c0769f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f072c000f60 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:49.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.483+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f072801f020 con 0x7f073815df10 2024-04-25T19:03:49.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.684+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0738074190 con 0x7f073815df10 2024-04-25T19:03:49.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.690+0000 7f0735600640 1 -- 172.21.15.123:0/3293187194 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f0738157670 con 0x7f073815df10 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 msgr2=0x7f071c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 0x7f071c0769f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f072c000f60 tx=0x7f072c009040 comp rx=0 tx=0).stop 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 msgr2=0x7f07381f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f072800b3f0 tx=0x7f072800b8b0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 shutdown_connections 2024-04-25T19:03:49.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f071c0744e0 0x7f071c0769f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0738156d20 0x7f0738157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f073815df10 0x7f07381f5be0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 --2- 172.21.15.123:0/3293187194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07380b0700 0x7f0738154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 >> 172.21.15.123:0/3293187194 conn(0x7f0738073090 msgr2=0x7f07380b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 shutdown_connections 2024-04-25T19:03:49.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:49.693+0000 7f073e800640 1 -- 172.21.15.123:0/3293187194 wait complete. 2024-04-25T19:03:49.725 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T19:03:49.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T19:03:49.850 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T19:03:49.963 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T19:03:49.964 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:49.964 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:50.023 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:50.023 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:50.076 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:50.076 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:50.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:50.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.404+0000 7f06ef200640 1 Processor -- start 2024-04-25T19:03:50.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.404+0000 7f06ef200640 1 -- start start 2024-04-25T19:03:50.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ef200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 0x7f06f015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ef200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f015f070 con 0x7f06f015e2f0 2024-04-25T19:03:50.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ef200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f006f340 con 0x7f06f006d580 2024-04-25T19:03:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ef200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f06f015ee10 con 0x7f06f006fe20 2024-04-25T19:03:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ee800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f06f015e2f0 0x7f06f015e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33736/0 (socket says 172.21.15.123:33736) 2024-04-25T19:03:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ee800640 1 -- 172.21.15.123:0/2622262742 learned_addr learned my addr 172.21.15.123:0/2622262742 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.406+0000 7f06ede00640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 0x7f06f015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720837753 0 0) 0x7f06f015f070 con 0x7f06f015e2f0 2024-04-25T19:03:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06d4003520 con 0x7f06f015e2f0 2024-04-25T19:03:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 506623645 0 0) 0x7f06d4003520 con 0x7f06f015e2f0 2024-04-25T19:03:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 msgr2=0x7f06f015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 0x7f06f015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 >> v1:172.21.15.146:6789/0 conn(0x7f06f006d580 legacy=0x7f06f006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f006f620 con 0x7f06f015e2f0 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f06e8001670 con 0x7f06f015e2f0 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 >> v1:172.21.15.123:6789/0 conn(0x7f06f015e2f0 legacy=0x7f06f015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 0x7f06d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d4003fb0 0x7f06d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4004d40 0x7f06d40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f015ee10 con 0x7f06d4003fb0 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f06d400bbb0 con 0x7f06d4004d40 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.407+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f06d400bdb0 con 0x7f06f006fe20 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d4003fb0 0x7f06d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 -- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 msgr2=0x7f06d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f006fe20 0x7f06d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 -- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4004d40 msgr2=0x7f06d40051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4004d40 0x7f06d40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.408+0000 7f06ed400640 1 -- 172.21.15.123:0/2622262742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d4003520 con 0x7f06d4003fb0 2024-04-25T19:03:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.409+0000 7f06ed400640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d4003fb0 0x7f06d4004800 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06d8001720 tx=0x7f06d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bd15d400d58b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.409+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8001370 con 0x7f06d4003fb0 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.409+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d80052d0 con 0x7f06d4003fb0 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.409+0000 7f06eca00640 1 -- 172.21.15.123:0/2622262742 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d802abc0 con 0x7f06d4003fb0 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 -- 172.21.15.123:0/2622262742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d4003fb0 msgr2=0x7f06d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 --2- 172.21.15.123:0/2622262742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d4003fb0 0x7f06d4004800 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06d8001720 tx=0x7f06d4005830 comp rx=0 tx=0).stop 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ee800640 1 -- 172.21.15.123:0/2622262742 reap_dead start 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 -- 172.21.15.123:0/2622262742 shutdown_connections 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 -- 172.21.15.123:0/2622262742 >> 172.21.15.123:0/2622262742 conn(0x7f06f0073200 msgr2=0x7f06f0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 -- 172.21.15.123:0/2622262742 shutdown_connections 2024-04-25T19:03:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.410+0000 7f06ef200640 1 -- 172.21.15.123:0/2622262742 wait complete. 2024-04-25T19:03:50.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.413+0000 7f06ef200640 1 Processor -- start 2024-04-25T19:03:50.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.413+0000 7f06ef200640 1 -- start start 2024-04-25T19:03:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4003fb0 0x7f06f01579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f0138ef0 0x7f06f0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f015b830 con 0x7f06f01f7830 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f06f015b6b0 con 0x7f06d4003fb0 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.414+0000 7f06ef200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f06f015b9b0 con 0x7f06f0138ef0 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56392/0 (socket says 172.21.15.123:56392) 2024-04-25T19:03:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 -- 172.21.15.123:0/2108136215 learned_addr learned my addr 172.21.15.123:0/2108136215 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ede00640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4003fb0 0x7f06f01579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 -- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f0138ef0 msgr2=0x7f06f0155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f0138ef0 0x7f06f0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 -- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4003fb0 msgr2=0x7f06f01579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4003fb0 0x7f06f01579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.415+0000 7f06ee800640 1 -- 172.21.15.123:0/2108136215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f01f8180 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06ee800640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004780 tx=0x7f06e800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8007440 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e80075e0 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004a20 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f0139300 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.416+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01582d0 con 0x7f06f01f7830 2024-04-25T19:03:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.417+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f0050740 con 0x7f06f01f7830 2024-04-25T19:03:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.418+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f06e8004bc0 con 0x7f06f01f7830 2024-04-25T19:03:50.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.420+0000 7f06e6000640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 0x7f06cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.420+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f06e8094c90 con 0x7f06f01f7830 2024-04-25T19:03:50.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.420+0000 7f06ede00640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 0x7f06cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.421+0000 7f06ede00640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 0x7f06cc0769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06e0004b30 tx=0x7f06e000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.425+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06e801b020 con 0x7f06f01f7830 2024-04-25T19:03:50.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.624+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06f0158a10 con 0x7f06f01f7830 2024-04-25T19:03:50.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.627+0000 7f06e6000640 1 -- 172.21.15.123:0/2108136215 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f06f01f8180 con 0x7f06f01f7830 2024-04-25T19:03:50.629 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:50.629 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 msgr2=0x7f06cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 0x7f06cc0769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06e0004b30 tx=0x7f06e000b040 comp rx=0 tx=0).stop 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 msgr2=0x7f06f01f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004780 tx=0x7f06e800c960 comp rx=0 tx=0).stop 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 shutdown_connections 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f06cc0744b0 0x7f06cc0769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f01f7830 0x7f06f01f7c40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06f0138ef0 0x7f06f0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 --2- 172.21.15.123:0/2108136215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06d4003fb0 0x7f06f01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 >> 172.21.15.123:0/2108136215 conn(0x7f06f0073200 msgr2=0x7f06f015f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 shutdown_connections 2024-04-25T19:03:50.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.630+0000 7f06ef200640 1 -- 172.21.15.123:0/2108136215 wait complete. 2024-04-25T19:03:50.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:50.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:50.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.931+0000 7f1ceb800640 1 Processor -- start 2024-04-25T19:03:50.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.931+0000 7f1ceb800640 1 -- start start 2024-04-25T19:03:50.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.933+0000 7f1ceb800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 0x7f1ce40cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.933+0000 7f1ceb800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ce4070820 0x7f1ce41609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.933+0000 7f1ceb800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce40cf300 con 0x7f1ce40cdca0 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.933+0000 7f1ceb800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce40cff10 con 0x7f1ce4070820 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.933+0000 7f1ceb800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce40cf5e0 con 0x7f1ce40d03a0 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ce4070820 0x7f1ce41609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ce4070820 0x7f1ce41609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56396/0 (socket says 172.21.15.123:56396) 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 -- 172.21.15.123:0/4167544149 learned_addr learned my addr 172.21.15.123:0/4167544149 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:50.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce8e00640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 0x7f1ce40cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 -- 172.21.15.123:0/4167544149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 msgr2=0x7f1ce40cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 0x7f1ce40cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 -- 172.21.15.123:0/4167544149 >> v1:172.21.15.146:6789/0 conn(0x7f1ce40cdca0 legacy=0x7f1ce415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce3e00640 1 -- 172.21.15.123:0/4167544149 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40cf8c0 con 0x7f1ce4070820 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.934+0000 7f1ce8e00640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 0x7f1ce40cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:50.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.935+0000 7f1ce3e00640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce41609e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009870 tx=0x7f1cd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6a7af369b282a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.936+0000 7f1ce3400640 1 -- 172.21.15.123:0/4167544149 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0037070 con 0x7f1ce4070820 2024-04-25T19:03:50.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.936+0000 7f1ce3400640 1 -- 172.21.15.123:0/4167544149 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0032440 con 0x7f1ce4070820 2024-04-25T19:03:50.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.936+0000 7f1ce3400640 1 -- 172.21.15.123:0/4167544149 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0032810 con 0x7f1ce4070820 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.936+0000 7f1ceb800640 1 -- 172.21.15.123:0/4167544149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 msgr2=0x7f1ce41609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.936+0000 7f1ceb800640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce41609e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009870 tx=0x7f1cd002a270 comp rx=0 tx=0).stop 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 -- 172.21.15.123:0/4167544149 shutdown_connections 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce41609e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 --2- 172.21.15.123:0/4167544149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40d03a0 0x7f1ce40cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 -- 172.21.15.123:0/4167544149 >> 172.21.15.123:0/4167544149 conn(0x7f1ce40ce390 msgr2=0x7f1ce40ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 -- 172.21.15.123:0/4167544149 shutdown_connections 2024-04-25T19:03:50.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.937+0000 7f1ceb800640 1 -- 172.21.15.123:0/4167544149 wait complete. 2024-04-25T19:03:50.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.939+0000 7f1ceb800640 1 Processor -- start 2024-04-25T19:03:50.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.940+0000 7f1ceb800640 1 -- start start 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40cdca0 0x7f1ce40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce40d03a0 0x7f1ce41e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4160f20 con 0x7f1ce4070820 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40cc2c0 con 0x7f1ce40d03a0 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.941+0000 7f1ceb800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce41610a0 con 0x7f1ce40cdca0 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56400/0 (socket says 172.21.15.123:56400) 2024-04-25T19:03:50.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 -- 172.21.15.123:0/3852246019 learned_addr learned my addr 172.21.15.123:0/3852246019 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce3e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40cdca0 0x7f1ce40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 -- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40cdca0 msgr2=0x7f1ce40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce9800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce40d03a0 0x7f1ce41e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40cdca0 0x7f1ce40b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 -- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce40d03a0 msgr2=0x7f1ce41e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce40d03a0 0x7f1ce41e7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 -- 172.21.15.123:0/3852246019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40b00e0 con 0x7f1ce4070820 2024-04-25T19:03:50.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.942+0000 7f1ce8e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800d800 tx=0x7f1cd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.943+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8009640 con 0x7f1ce4070820 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.943+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd8019070 con 0x7f1ce4070820 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.943+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce41e6a70 con 0x7f1ce4070820 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.943+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8009a20 con 0x7f1ce4070820 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.943+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce41e7030 con 0x7f1ce4070820 2024-04-25T19:03:50.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.944+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce4076a40 con 0x7f1ce4070820 2024-04-25T19:03:50.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.945+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1cd8009bc0 con 0x7f1ce4070820 2024-04-25T19:03:50.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.946+0000 7f1ce0c00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 0x7f1cc40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.946+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1cd80958e0 con 0x7f1ce4070820 2024-04-25T19:03:50.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.947+0000 7f1ce3e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 0x7f1cc40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.947+0000 7f1ce3e00640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 0x7f1cc40769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0034000 tx=0x7f1cd002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:50.951+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cd805fd90 con 0x7f1ce4070820 2024-04-25T19:03:51.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.155+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ce4069c60 con 0x7f1ce4070820 2024-04-25T19:03:51.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.160+0000 7f1ce0c00640 1 -- 172.21.15.123:0/3852246019 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f1cd805f690 con 0x7f1ce4070820 2024-04-25T19:03:51.162 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 msgr2=0x7f1cc40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 0x7f1cc40769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0034000 tx=0x7f1cd002e000 comp rx=0 tx=0).stop 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 msgr2=0x7f1ce414e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800d800 tx=0x7f1cd800dcc0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 shutdown_connections 2024-04-25T19:03:51.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1cc40744e0 0x7f1cc40769f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce40d03a0 0x7f1ce41e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce40cdca0 0x7f1ce40b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 --2- 172.21.15.123:0/3852246019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4070820 0x7f1ce414e740 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 >> 172.21.15.123:0/3852246019 conn(0x7f1ce40ce390 msgr2=0x7f1ce415aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 shutdown_connections 2024-04-25T19:03:51.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.163+0000 7f1ceb800640 1 -- 172.21.15.123:0/3852246019 wait complete. 2024-04-25T19:03:51.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:51.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.459+0000 7f7da9a00640 1 Processor -- start 2024-04-25T19:03:51.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.459+0000 7f7da9a00640 1 -- start start 2024-04-25T19:03:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.461+0000 7f7da9a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7da406daa0 0x7f7da415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.461+0000 7f7da9a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da40cd140 0x7f7da40af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.461+0000 7f7da9a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da40cc5f0 con 0x7f7da4162540 2024-04-25T19:03:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.461+0000 7f7da9a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7da406deb0 con 0x7f7da406daa0 2024-04-25T19:03:51.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.461+0000 7f7da9a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7da406d610 con 0x7f7da40cd140 2024-04-25T19:03:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7da406daa0 0x7f7da415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7da4162540 0x7f7da40cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33742/0 (socket says 172.21.15.123:33742) 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7da406daa0 0x7f7da415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56404/0 (socket says 172.21.15.123:56404) 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2000640 1 -- 172.21.15.123:0/332295339 learned_addr learned my addr 172.21.15.123:0/332295339 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da3400640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da40cd140 0x7f7da40af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2a00640 1 -- 172.21.15.123:0/332295339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da40cd140 msgr2=0x7f7da40af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.462+0000 7f7da2a00640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da40cd140 0x7f7da40af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.463+0000 7f7da2a00640 1 -- 172.21.15.123:0/332295339 >> v1:172.21.15.123:6789/0 conn(0x7f7da4162540 legacy=0x7f7da40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.463+0000 7f7da1600640 1 -- 172.21.15.123:0/332295339 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962950752 0 0) 0x7f7da40cc5f0 con 0x7f7da4162540 2024-04-25T19:03:51.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.463+0000 7f7da2a00640 1 -- 172.21.15.123:0/332295339 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da406ca00 con 0x7f7da406daa0 2024-04-25T19:03:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.464+0000 7f7da2a00640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da406daa0 0x7f7da415dfc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7d940097a0 tx=0x7f7d9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b82176d45988e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.464+0000 7f7da1600640 1 -- 172.21.15.123:0/332295339 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94037070 con 0x7f7da406daa0 2024-04-25T19:03:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.464+0000 7f7da1600640 1 -- 172.21.15.123:0/332295339 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d94032440 con 0x7f7da406daa0 2024-04-25T19:03:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.464+0000 7f7da1600640 1 -- 172.21.15.123:0/332295339 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94032810 con 0x7f7da406daa0 2024-04-25T19:03:51.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 -- 172.21.15.123:0/332295339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da406daa0 msgr2=0x7f7da415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da406daa0 0x7f7da415dfc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7d940097a0 tx=0x7f7d9402a180 comp rx=0 tx=0).stop 2024-04-25T19:03:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 -- 172.21.15.123:0/332295339 shutdown_connections 2024-04-25T19:03:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da40cd140 0x7f7da40af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 --2- 172.21.15.123:0/332295339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da406daa0 0x7f7da415dfc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 -- 172.21.15.123:0/332295339 >> 172.21.15.123:0/332295339 conn(0x7f7da40ce390 msgr2=0x7f7da40ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 -- 172.21.15.123:0/332295339 shutdown_connections 2024-04-25T19:03:51.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.465+0000 7f7da9a00640 1 -- 172.21.15.123:0/332295339 wait complete. 2024-04-25T19:03:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.467+0000 7f7da9a00640 1 Processor -- start 2024-04-25T19:03:51.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.468+0000 7f7da9a00640 1 -- start start 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7da406daa0 0x7f7da414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da415e590 0x7f7da40cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7da406d2a0 con 0x7f7da4162540 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7da406cfc0 con 0x7f7da406daa0 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da9a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7da4162950 con 0x7f7da415e590 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56412/0 (socket says 172.21.15.123:56412) 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.469+0000 7f7da2000640 1 -- 172.21.15.123:0/2524639308 learned_addr learned my addr 172.21.15.123:0/2524639308 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da3400640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da415e590 0x7f7da40cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7da406daa0 0x7f7da414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 -- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da415e590 msgr2=0x7f7da40cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da415e590 0x7f7da40cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 -- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7da406daa0 msgr2=0x7f7da414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7da406daa0 0x7f7da414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 -- 172.21.15.123:0/2524639308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41610a0 con 0x7f7da4162540 2024-04-25T19:03:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7da2000640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c430 tx=0x7f7d9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:51.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.470+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900074e0 con 0x7f7da4162540 2024-04-25T19:03:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.471+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d900047e0 con 0x7f7da4162540 2024-04-25T19:03:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.471+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90004bd0 con 0x7f7da4162540 2024-04-25T19:03:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.471+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da4161380 con 0x7f7da4162540 2024-04-25T19:03:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.471+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da4160b30 con 0x7f7da4162540 2024-04-25T19:03:51.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.472+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da41618a0 con 0x7f7da4162540 2024-04-25T19:03:51.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.473+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d90011070 con 0x7f7da4162540 2024-04-25T19:03:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.474+0000 7f7d9aa00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 0x7f7d840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.474+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f7d90095210 con 0x7f7da4162540 2024-04-25T19:03:51.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.474+0000 7f7da2a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 0x7f7d840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.475+0000 7f7da2a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 0x7f7d840769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d94034000 tx=0x7f7d9402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:51.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.479+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d9005f590 con 0x7f7da4162540 2024-04-25T19:03:51.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.675+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7da4076a40 con 0x7f7da4162540 2024-04-25T19:03:51.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.676+0000 7f7d9aa00640 1 -- 172.21.15.123:0/2524639308 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f7d9005ee90 con 0x7f7da4162540 2024-04-25T19:03:51.678 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 msgr2=0x7f7d840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 0x7f7d840769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d94034000 tx=0x7f7d9402e000 comp rx=0 tx=0).stop 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 msgr2=0x7f7da40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c430 tx=0x7f7d9000c8f0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 shutdown_connections 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7d840744e0 0x7f7d840769f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7da415e590 0x7f7da40cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da4162540 0x7f7da40caa80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 --2- 172.21.15.123:0/2524639308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7da406daa0 0x7f7da414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 >> 172.21.15.123:0/2524639308 conn(0x7f7da40ce390 msgr2=0x7f7da415d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 shutdown_connections 2024-04-25T19:03:51.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.679+0000 7f7da9a00640 1 -- 172.21.15.123:0/2524639308 wait complete. 2024-04-25T19:03:51.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.985+0000 7fac28000640 1 Processor -- start 2024-04-25T19:03:51.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.985+0000 7fac28000640 1 -- start start 2024-04-25T19:03:51.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac28000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac2015e180 0x7fac200cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac28000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac200cc260 0x7fac20166d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac28000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac200cc670 con 0x7fac2015dd70 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac28000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fac200c7b60 con 0x7fac200cc260 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac28000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fac200c7ff0 con 0x7fac2015e180 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac26000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac200cc260 0x7fac20166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac26000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac200cc260 0x7fac20166d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56428/0 (socket says 172.21.15.123:56428) 2024-04-25T19:03:51.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac26000640 1 -- 172.21.15.123:0/2353643700 learned_addr learned my addr 172.21.15.123:0/2353643700 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.988+0000 7fac24c00640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac2015e180 0x7fac200cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.989+0000 7fac26000640 1 -- 172.21.15.123:0/2353643700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac2015e180 msgr2=0x7fac200cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.989+0000 7fac26000640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac2015e180 0x7fac200cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.989+0000 7fac26000640 1 -- 172.21.15.123:0/2353643700 >> v1:172.21.15.146:6789/0 conn(0x7fac2015dd70 legacy=0x7fac200c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:51.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.989+0000 7fac26000640 1 -- 172.21.15.123:0/2353643700 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac200c6ca0 con 0x7fac200cc260 2024-04-25T19:03:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.990+0000 7fac26000640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac200cc260 0x7fac20166d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ababfcdaa21da3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.990+0000 7fac1fe00640 1 -- 172.21.15.123:0/2353643700 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18037070 con 0x7fac200cc260 2024-04-25T19:03:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.990+0000 7fac1fe00640 1 -- 172.21.15.123:0/2353643700 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18032440 con 0x7fac200cc260 2024-04-25T19:03:51.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.990+0000 7fac1fe00640 1 -- 172.21.15.123:0/2353643700 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18032810 con 0x7fac200cc260 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 -- 172.21.15.123:0/2353643700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac200cc260 msgr2=0x7fac20166d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac200cc260 0x7fac20166d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1802a270 comp rx=0 tx=0).stop 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 -- 172.21.15.123:0/2353643700 shutdown_connections 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac200cc260 0x7fac20166d90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 --2- 172.21.15.123:0/2353643700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac2015e180 0x7fac200cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 -- 172.21.15.123:0/2353643700 >> 172.21.15.123:0/2353643700 conn(0x7fac200ce150 msgr2=0x7fac200ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 -- 172.21.15.123:0/2353643700 shutdown_connections 2024-04-25T19:03:51.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.991+0000 7fac28000640 1 -- 172.21.15.123:0/2353643700 wait complete. 2024-04-25T19:03:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.993+0000 7fac28000640 1 Processor -- start 2024-04-25T19:03:51.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.994+0000 7fac28000640 1 -- start start 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac200cc260 0x7fac2015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac2015dd70 0x7fac2009c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac200c7510 con 0x7fac2015e180 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac200c7230 con 0x7fac2015dd70 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac28000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fac200c77f0 con 0x7fac200cc260 2024-04-25T19:03:51.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.995+0000 7fac26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56432/0 (socket says 172.21.15.123:56432) 2024-04-25T19:03:51.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 -- 172.21.15.123:0/88297617 learned_addr learned my addr 172.21.15.123:0/88297617 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac24c00640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac2015dd70 0x7fac2009c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac25600640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac200cc260 0x7fac2015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 -- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac200cc260 msgr2=0x7fac2015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac200cc260 0x7fac2015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 -- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac2015dd70 msgr2=0x7fac2009c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac2015dd70 0x7fac2009c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 -- 172.21.15.123:0/88297617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac20156b00 con 0x7fac2015e180 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.996+0000 7fac26000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.997+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18037040 con 0x7fac2015e180 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.997+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18042070 con 0x7fac2015e180 2024-04-25T19:03:51.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.997+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18030920 con 0x7fac2015e180 2024-04-25T19:03:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.997+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac200c87e0 con 0x7fac2015e180 2024-04-25T19:03:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.997+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac200c8da0 con 0x7fac2015e180 2024-04-25T19:03:52.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:51.998+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac20076a40 con 0x7fac2015e180 2024-04-25T19:03:52.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.000+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac18030ac0 con 0x7fac2015e180 2024-04-25T19:03:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.001+0000 7fac1d600640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 0x7fac000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.001+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fac180b5ee0 con 0x7fac2015e180 2024-04-25T19:03:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.001+0000 7fac25600640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 0x7fac000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.002+0000 7fac25600640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 0x7fac000769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac200d15f0 tx=0x7fac14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.005+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac18080390 con 0x7fac2015e180 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.205+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac200705d0 con 0x7fac2015e180 2024-04-25T19:03:52.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.206+0000 7fac1d600640 1 -- 172.21.15.123:0/88297617 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fac1807fc90 con 0x7fac2015e180 2024-04-25T19:03:52.208 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 msgr2=0x7fac000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 0x7fac000769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac200d15f0 tx=0x7fac14009040 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 msgr2=0x7fac201565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac180046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 shutdown_connections 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fac000744e0 0x7fac000769f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2015e180 0x7fac201565c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac2015dd70 0x7fac2009c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 --2- 172.21.15.123:0/88297617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac200cc260 0x7fac2015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 >> 172.21.15.123:0/88297617 conn(0x7fac200ce150 msgr2=0x7fac20070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 shutdown_connections 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.209+0000 7fac28000640 1 -- 172.21.15.123:0/88297617 wait complete. 2024-04-25T19:03:52.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.497+0000 7fef5e000640 1 Processor -- start 2024-04-25T19:03:52.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.498+0000 7fef5e000640 1 -- start start 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef5814da70 0x7fef58164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5e000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef5806dc90 0x7fef580b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5e000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef581653e0 con 0x7fef58164e60 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5e000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fef580b44f0 con 0x7fef5814da70 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5e000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fef581656c0 con 0x7fef5806dc90 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5cc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef5814da70 0x7fef58164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5cc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef5814da70 0x7fef58164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56444/0 (socket says 172.21.15.123:56444) 2024-04-25T19:03:52.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.500+0000 7fef5cc00640 1 -- 172.21.15.123:0/3044086165 learned_addr learned my addr 172.21.15.123:0/3044086165 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:52.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.501+0000 7fef5cc00640 1 -- 172.21.15.123:0/3044086165 >> v2:172.21.15.146:3300/0 conn(0x7fef5806dc90 msgr2=0x7fef580b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.501+0000 7fef5cc00640 1 --2- 172.21.15.123:0/3044086165 >> v2:172.21.15.146:3300/0 conn(0x7fef5806dc90 0x7fef580b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.501+0000 7fef5cc00640 1 -- 172.21.15.123:0/3044086165 >> v1:172.21.15.146:6789/0 conn(0x7fef58164e60 legacy=0x7fef5806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.501+0000 7fef5cc00640 1 -- 172.21.15.123:0/3044086165 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580b4740 con 0x7fef5814da70 2024-04-25T19:03:52.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.502+0000 7fef5cc00640 1 --2- 172.21.15.123:0/3044086165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5814da70 0x7fef58164920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef58070b00 tx=0x7fef4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c709a2897704c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.502+0000 7fef57e00640 1 -- 172.21.15.123:0/3044086165 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48037070 con 0x7fef5814da70 2024-04-25T19:03:52.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.502+0000 7fef57e00640 1 -- 172.21.15.123:0/3044086165 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef48032440 con 0x7fef5814da70 2024-04-25T19:03:52.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.502+0000 7fef57e00640 1 -- 172.21.15.123:0/3044086165 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48032810 con 0x7fef5814da70 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 -- 172.21.15.123:0/3044086165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5814da70 msgr2=0x7fef58164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 --2- 172.21.15.123:0/3044086165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5814da70 0x7fef58164920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef58070b00 tx=0x7fef4802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 -- 172.21.15.123:0/3044086165 shutdown_connections 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 --2- 172.21.15.123:0/3044086165 >> v2:172.21.15.146:3300/0 conn(0x7fef5806dc90 0x7fef580b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 --2- 172.21.15.123:0/3044086165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5814da70 0x7fef58164920 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 -- 172.21.15.123:0/3044086165 >> 172.21.15.123:0/3044086165 conn(0x7fef58073090 msgr2=0x7fef580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 -- 172.21.15.123:0/3044086165 shutdown_connections 2024-04-25T19:03:52.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.503+0000 7fef5e000640 1 -- 172.21.15.123:0/3044086165 wait complete. 2024-04-25T19:03:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.505+0000 7fef5e000640 1 Processor -- start 2024-04-25T19:03:52.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.505+0000 7fef5e000640 1 -- start start 2024-04-25T19:03:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef5814da70 0x7fef581670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5815ecd0 0x7fef5815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef580704b0 con 0x7fef58164e60 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fef5814de80 con 0x7fef5814da70 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5e000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef58070630 con 0x7fef5815ecd0 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56450/0 (socket says 172.21.15.123:56450) 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef4fe00640 1 -- 172.21.15.123:0/3016318871 learned_addr learned my addr 172.21.15.123:0/3016318871 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.507+0000 7fef5cc00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef5814da70 0x7fef581670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 -- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5815ecd0 msgr2=0x7fef5815f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5815ecd0 0x7fef5815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 -- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef5814da70 msgr2=0x7fef581670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef5814da70 0x7fef581670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 -- 172.21.15.123:0/3016318871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5815f620 con 0x7fef58164e60 2024-04-25T19:03:52.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef4fe00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b420 tx=0x7fef4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40011830 con 0x7fef58164e60 2024-04-25T19:03:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef40016070 con 0x7fef58164e60 2024-04-25T19:03:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef580b0be0 con 0x7fef58164e60 2024-04-25T19:03:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40011bf0 con 0x7fef58164e60 2024-04-25T19:03:52.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.508+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581668e0 con 0x7fef58164e60 2024-04-25T19:03:52.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.509+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef58050740 con 0x7fef58164e60 2024-04-25T19:03:52.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.511+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef4001b460 con 0x7fef58164e60 2024-04-25T19:03:52.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.511+0000 7fef55600640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 0x7fef340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.512+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fef40095360 con 0x7fef58164e60 2024-04-25T19:03:52.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.512+0000 7fef5cc00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 0x7fef340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.512+0000 7fef5cc00640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 0x7fef340769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef48034000 tx=0x7fef4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.516+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef4005f6e0 con 0x7fef58164e60 2024-04-25T19:03:52.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.714+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fef580d0f90 con 0x7fef58164e60 2024-04-25T19:03:52.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.715+0000 7fef55600640 1 -- 172.21.15.123:0/3016318871 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fef4005efe0 con 0x7fef58164e60 2024-04-25T19:03:52.717 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 msgr2=0x7fef340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 0x7fef340769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef48034000 tx=0x7fef4802e000 comp rx=0 tx=0).stop 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 msgr2=0x7fef581491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b420 tx=0x7fef4000b8e0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 shutdown_connections 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fef340744e0 0x7fef340769f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5815ecd0 0x7fef5815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef58164e60 0x7fef581491c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 --2- 172.21.15.123:0/3016318871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef5814da70 0x7fef581670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 >> 172.21.15.123:0/3016318871 conn(0x7fef58073090 msgr2=0x7fef5814c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 shutdown_connections 2024-04-25T19:03:52.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:52.718+0000 7fef5e000640 1 -- 172.21.15.123:0/3016318871 wait complete. 2024-04-25T19:03:52.745 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:03:52.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:03:53.002 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:03:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.020+0000 7fd351e00640 1 Processor -- start 2024-04-25T19:03:53.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.021+0000 7fd351e00640 1 -- start start 2024-04-25T19:03:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.022+0000 7fd351e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd34c0b4b30 0x7fd34c0d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.022+0000 7fd351e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c160480 con 0x7fd34c160070 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.022+0000 7fd351e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c06c710 con 0x7fd34c0d0750 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.022+0000 7fd351e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c06c9f0 con 0x7fd34c0b4b30 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd34c0b4b30 0x7fd34c0d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd34c0b4b30 0x7fd34c0d4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56454/0 (socket says 172.21.15.123:56454) 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 -- 172.21.15.123:0/258176536 learned_addr learned my addr 172.21.15.123:0/258176536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd34be00640 1 --1- 172.21.15.123:0/258176536 >> v1:172.21.15.123:6789/0 conn(0x7fd34c160070 0x7fd34c0b8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33752/0 (socket says 172.21.15.123:33752) 2024-04-25T19:03:53.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 -- 172.21.15.123:0/258176536 >> v1:172.21.15.146:6789/0 conn(0x7fd34c0d0750 legacy=0x7fd34c0d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 -- 172.21.15.123:0/258176536 >> v1:172.21.15.123:6789/0 conn(0x7fd34c160070 legacy=0x7fd34c0b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd350a00640 1 -- 172.21.15.123:0/258176536 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c06ccd0 con 0x7fd34c0b4b30 2024-04-25T19:03:53.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.023+0000 7fd34b400640 1 -- 172.21.15.123:0/258176536 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040998973 0 0) 0x7fd34c160480 con 0x7fd34c160070 2024-04-25T19:03:53.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.024+0000 7fd350a00640 1 --2- 172.21.15.123:0/258176536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c0b4b30 0x7fd34c0d4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efadbd81b035d3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.025+0000 7fd34b400640 1 -- 172.21.15.123:0/258176536 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340037070 con 0x7fd34c0b4b30 2024-04-25T19:03:53.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.025+0000 7fd34b400640 1 -- 172.21.15.123:0/258176536 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd340032440 con 0x7fd34c0b4b30 2024-04-25T19:03:53.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.025+0000 7fd34b400640 1 -- 172.21.15.123:0/258176536 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340032810 con 0x7fd34c0b4b30 2024-04-25T19:03:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 -- 172.21.15.123:0/258176536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c0b4b30 msgr2=0x7fd34c0d4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 --2- 172.21.15.123:0/258176536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c0b4b30 0x7fd34c0d4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34002a270 comp rx=0 tx=0).stop 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 -- 172.21.15.123:0/258176536 shutdown_connections 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 --2- 172.21.15.123:0/258176536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c0b4b30 0x7fd34c0d4d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 -- 172.21.15.123:0/258176536 >> 172.21.15.123:0/258176536 conn(0x7fd34c0d65d0 msgr2=0x7fd34c0b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 -- 172.21.15.123:0/258176536 shutdown_connections 2024-04-25T19:03:53.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.026+0000 7fd351e00640 1 -- 172.21.15.123:0/258176536 wait complete. 2024-04-25T19:03:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.029+0000 7fd351e00640 1 Processor -- start 2024-04-25T19:03:53.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.029+0000 7fd351e00640 1 -- start start 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd34c0b4b30 0x7fd34c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34c0d0750 0x7fd34c159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0b95e0 con 0x7fd34c160070 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0d1410 con 0x7fd34c0d0750 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.030+0000 7fd351e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c06d4d0 con 0x7fd34c0b4b30 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56460/0 (socket says 172.21.15.123:56460) 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 -- 172.21.15.123:0/335580730 learned_addr learned my addr 172.21.15.123:0/335580730 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:53.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd350a00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd34c0b4b30 0x7fd34c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd34be00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34c0d0750 0x7fd34c159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 -- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd34c0b4b30 msgr2=0x7fd34c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd34c0b4b30 0x7fd34c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 -- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34c0d0750 msgr2=0x7fd34c159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34c0d0750 0x7fd34c159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.031+0000 7fd351400640 1 -- 172.21.15.123:0/335580730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c154700 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd351400640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c6e0 tx=0x7fd34400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344003b70 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344003d30 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c0cab90 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344005310 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.032+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c0cb150 con 0x7fd34c160070 2024-04-25T19:03:53.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.034+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c050740 con 0x7fd34c160070 2024-04-25T19:03:53.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.035+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3440054b0 con 0x7fd34c160070 2024-04-25T19:03:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.036+0000 7fd348c00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 0x7fd32c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.036+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fd344095400 con 0x7fd34c160070 2024-04-25T19:03:53.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.037+0000 7fd350a00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 0x7fd32c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.037+0000 7fd350a00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 0x7fd32c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd340034000 tx=0x7fd34002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.041+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd34c154700 con 0x7fd34c160070 2024-04-25T19:03:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.252+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd34c0ca920 con 0x7fd34c160070 2024-04-25T19:03:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.253+0000 7fd348c00640 1 -- 172.21.15.123:0/335580730 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fd34405f790 con 0x7fd34c160070 2024-04-25T19:03:53.254 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:03:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.255+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 msgr2=0x7fd32c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.255+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 0x7fd32c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd340034000 tx=0x7fd34002e000 comp rx=0 tx=0).stop 2024-04-25T19:03:53.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.255+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 msgr2=0x7fd34c159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.255+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c6e0 tx=0x7fd34400cba0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.255+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 shutdown_connections 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd32c0744b0 0x7fd32c0769c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c160070 0x7fd34c159e80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34c0d0750 0x7fd34c159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 --2- 172.21.15.123:0/335580730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd34c0b4b30 0x7fd34c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 >> 172.21.15.123:0/335580730 conn(0x7fd34c0d65d0 msgr2=0x7fd34c15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 shutdown_connections 2024-04-25T19:03:53.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.256+0000 7fd351e00640 1 -- 172.21.15.123:0/335580730 wait complete. 2024-04-25T19:03:53.286 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T19:03:53.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:53.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T19:03:53.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:03:53.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:53.363 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T19:03:53.407 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:03:53.407 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:53.466 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:03:53.466 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:53.516 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:03:53.517 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:53.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.826+0000 7f5c7d600640 1 Processor -- start 2024-04-25T19:03:53.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.826+0000 7f5c7d600640 1 -- start start 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c7d600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c7806d580 0x7f5c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c7d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c7815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c7d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7806ded0 con 0x7f5c7806fe20 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c7d600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7815f070 con 0x7f5c7806d580 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c7d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7806f340 con 0x7f5c7815e2f0 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c6f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c7806d580 0x7f5c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c77e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5c7806fe20 0x7f5c7815b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33762/0 (socket says 172.21.15.123:33762) 2024-04-25T19:03:53.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.828+0000 7f5c77e00640 1 -- 172.21.15.123:0/1842352871 learned_addr learned my addr 172.21.15.123:0/1842352871 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:53.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.829+0000 7f5c7cc00640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c7815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.829+0000 7f5c6f400640 1 -- 172.21.15.123:0/1842352871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 msgr2=0x7f5c7815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:53.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.829+0000 7f5c6f400640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c7815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.829+0000 7f5c6f400640 1 -- 172.21.15.123:0/1842352871 >> v1:172.21.15.123:6789/0 conn(0x7f5c7806fe20 legacy=0x7f5c7815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.829+0000 7f5c6f400640 1 -- 172.21.15.123:0/1842352871 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7806f620 con 0x7f5c7806d580 2024-04-25T19:03:53.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.830+0000 7f5c6f400640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c7806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c600097a0 tx=0x7f5c6002e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ecb3a6d85e2e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.830+0000 7f5c77400640 1 -- 172.21.15.123:0/1842352871 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6003a070 con 0x7f5c7806d580 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.830+0000 7f5c77400640 1 -- 172.21.15.123:0/1842352871 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c60033710 con 0x7f5c7806d580 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.830+0000 7f5c77400640 1 -- 172.21.15.123:0/1842352871 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60033ae0 con 0x7f5c7806d580 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 -- 172.21.15.123:0/1842352871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 msgr2=0x7f5c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c7806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c600097a0 tx=0x7f5c6002e510 comp rx=0 tx=0).stop 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 -- 172.21.15.123:0/1842352871 shutdown_connections 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c7815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 --2- 172.21.15.123:0/1842352871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c7806d990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 -- 172.21.15.123:0/1842352871 >> 172.21.15.123:0/1842352871 conn(0x7f5c78073200 msgr2=0x7f5c78073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 -- 172.21.15.123:0/1842352871 shutdown_connections 2024-04-25T19:03:53.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.831+0000 7f5c7d600640 1 -- 172.21.15.123:0/1842352871 wait complete. 2024-04-25T19:03:53.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.835+0000 7f5c7d600640 1 Processor -- start 2024-04-25T19:03:53.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.835+0000 7f5c7d600640 1 -- start start 2024-04-25T19:03:53.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7806fe20 0x7f5c781f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c780b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7815edc0 con 0x7f5c7806d580 2024-04-25T19:03:53.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7815ec40 con 0x7f5c7806fe20 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c7d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5c780ce500 con 0x7f5c7815e2f0 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56472/0 (socket says 172.21.15.123:56472) 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 -- 172.21.15.123:0/2432768834 learned_addr learned my addr 172.21.15.123:0/2432768834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 -- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 msgr2=0x7f5c780b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c780b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 -- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7806fe20 msgr2=0x7f5c781f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7806fe20 0x7f5c781f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.838+0000 7f5c77e00640 1 -- 172.21.15.123:0/2432768834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c78142660 con 0x7f5c7806d580 2024-04-25T19:03:53.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c77e00640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3f0 tx=0x7f5c6800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68017070 con 0x7f5c7806d580 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c68005ce0 con 0x7f5c7806d580 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680072f0 con 0x7f5c7806d580 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c78142940 con 0x7f5c7806d580 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.839+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c78142f00 con 0x7f5c7806d580 2024-04-25T19:03:53.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.840+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c780d10e0 con 0x7f5c7806d580 2024-04-25T19:03:53.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.842+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c6801a070 con 0x7f5c7806d580 2024-04-25T19:03:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.843+0000 7f5c74c00640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 0x7f5c540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.843+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f5c68094830 con 0x7f5c7806d580 2024-04-25T19:03:53.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.843+0000 7f5c6f400640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 0x7f5c540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:53.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.844+0000 7f5c6f400640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 0x7f5c540769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c60037000 tx=0x7f5c60032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:53.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:53.847+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c7806ded0 con 0x7f5c7806d580 2024-04-25T19:03:54.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.050+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c780aa520 con 0x7f5c7806d580 2024-04-25T19:03:54.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.053+0000 7f5c74c00640 1 -- 172.21.15.123:0/2432768834 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f5c6805ece0 con 0x7f5c7806d580 2024-04-25T19:03:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:54.055 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:54.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.055+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 msgr2=0x7f5c540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.055+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 0x7f5c540769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c60037000 tx=0x7f5c60032000 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.055+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 msgr2=0x7f5c781413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.055+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3f0 tx=0x7f5c6800c7f0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 shutdown_connections 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f5c540744e0 0x7f5c540769f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c7815e2f0 0x7f5c780b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c7806fe20 0x7f5c781f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 --2- 172.21.15.123:0/2432768834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c7806d580 0x7f5c781413a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 >> 172.21.15.123:0/2432768834 conn(0x7f5c78073200 msgr2=0x7f5c78161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 shutdown_connections 2024-04-25T19:03:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.056+0000 7f5c7d600640 1 -- 172.21.15.123:0/2432768834 wait complete. 2024-04-25T19:03:54.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:54.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:54.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.341+0000 7fb9a9e00640 1 Processor -- start 2024-04-25T19:03:54.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.342+0000 7fb9a9e00640 1 -- start start 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a9e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb9a415d520 0x7fb9a406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a9e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a406ded0 con 0x7fb9a40d0e50 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a9e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a4070840 con 0x7fb9a406dac0 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a9e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a415d930 con 0x7fb9a415d520 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a8a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb9a40d0e50 0x7fb9a41608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33774/0 (socket says 172.21.15.123:33774) 2024-04-25T19:03:54.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.344+0000 7fb9a8a00640 1 -- 172.21.15.123:0/3343442166 learned_addr learned my addr 172.21.15.123:0/3343442166 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:54.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1682179034 0 0) 0x7fb998001110 con 0x7fb9a40d0e50 2024-04-25T19:03:54.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3e00640 1 --2- 172.21.15.123:0/3343442166 >> v2:172.21.15.146:3300/0 conn(0x7fb9a415d520 0x7fb9a406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb98c003520 con 0x7fb9a40d0e50 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120592073 0 0) 0x7fb98c003520 con 0x7fb9a40d0e50 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 >> v2:172.21.15.146:3300/0 conn(0x7fb9a415d520 msgr2=0x7fb9a406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 --2- 172.21.15.123:0/3343442166 >> v2:172.21.15.146:3300/0 conn(0x7fb9a415d520 0x7fb9a406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 >> v1:172.21.15.146:6789/0 conn(0x7fb9a406dac0 legacy=0x7fb9a40b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a40b67a0 con 0x7fb9a40d0e50 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fb998001840 con 0x7fb9a40d0e50 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 >> v1:172.21.15.123:6789/0 conn(0x7fb9a40d0e50 legacy=0x7fb9a41608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb98c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb98c004800 0x7fb98c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.345+0000 7fb9a3400640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb98c005270 0x7fb98c00cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a415d930 con 0x7fb98c0039b0 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c00d310 con 0x7fb98c005270 2024-04-25T19:03:54.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c005a10 con 0x7fb98c004800 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb98c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a3e00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb98c004800 0x7fb98c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a9400640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb98c005270 0x7fb98c00cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 -- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb98c004800 msgr2=0x7fb98c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb98c004800 0x7fb98c004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 -- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb98c005270 msgr2=0x7fb98c00cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb98c005270 0x7fb98c00cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a9400640 1 -- 172.21.15.123:0/3343442166 reap_dead start 2024-04-25T19:03:54.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.346+0000 7fb9a8a00640 1 -- 172.21.15.123:0/3343442166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c003520 con 0x7fb98c0039b0 2024-04-25T19:03:54.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.347+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb98c003e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c4c0 tx=0x7fb98c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a91fdf58329bf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.348+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99800c0e0 con 0x7fb98c0039b0 2024-04-25T19:03:54.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.348+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998034070 con 0x7fb98c0039b0 2024-04-25T19:03:54.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.348+0000 7fb9a3400640 1 -- 172.21.15.123:0/3343442166 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998005d70 con 0x7fb98c0039b0 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.348+0000 7fb9a9e00640 1 -- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 msgr2=0x7fb98c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.348+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb98c003e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c4c0 tx=0x7fb98c005830 comp rx=0 tx=0).stop 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.349+0000 7fb9a9e00640 1 -- 172.21.15.123:0/3343442166 shutdown_connections 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.349+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/3343442166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb98c003e20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.349+0000 7fb9a9e00640 1 -- 172.21.15.123:0/3343442166 >> 172.21.15.123:0/3343442166 conn(0x7fb9a4073090 msgr2=0x7fb9a40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.349+0000 7fb9a9e00640 1 -- 172.21.15.123:0/3343442166 shutdown_connections 2024-04-25T19:03:54.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.349+0000 7fb9a9e00640 1 -- 172.21.15.123:0/3343442166 wait complete. 2024-04-25T19:03:54.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.351+0000 7fb9a9e00640 1 Processor -- start 2024-04-25T19:03:54.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.351+0000 7fb9a9e00640 1 -- start start 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9a41f6780 0x7fb9a406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9a406d2a0 0x7fb9a4141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a415cd20 con 0x7fb98c0039b0 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a415ca40 con 0x7fb9a41f6780 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a415d000 con 0x7fb9a406d2a0 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56490/0 (socket says 172.21.15.123:56490) 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 -- 172.21.15.123:0/2224894400 learned_addr learned my addr 172.21.15.123:0/2224894400 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a3e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9a41f6780 0x7fb9a406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a9400640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9a406d2a0 0x7fb9a4141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 -- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9a406d2a0 msgr2=0x7fb9a4141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9a406d2a0 0x7fb9a4141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 -- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9a41f6780 msgr2=0x7fb9a406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9a41f6780 0x7fb9a406cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.353+0000 7fb9a8a00640 1 -- 172.21.15.123:0/2224894400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a406d6b0 con 0x7fb98c0039b0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a8a00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c5e0 tx=0x7fb998001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998007270 con 0x7fb98c0039b0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99803f030 con 0x7fb98c0039b0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a40a9f00 con 0x7fb98c0039b0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998007610 con 0x7fb98c0039b0 2024-04-25T19:03:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.354+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41f2e10 con 0x7fb98c0039b0 2024-04-25T19:03:54.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.356+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a40aa0d0 con 0x7fb98c0039b0 2024-04-25T19:03:54.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.356+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb99802f480 con 0x7fb98c0039b0 2024-04-25T19:03:54.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.357+0000 7fb9a0c00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 0x7fb9840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.358+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fb9980b6460 con 0x7fb98c0039b0 2024-04-25T19:03:54.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.358+0000 7fb9a3e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 0x7fb9840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.358+0000 7fb9a3e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 0x7fb9840769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb9a414fab0 tx=0x7fb990008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.362+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb998049020 con 0x7fb98c0039b0 2024-04-25T19:03:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.559+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9a4050740 con 0x7fb98c0039b0 2024-04-25T19:03:54.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.560+0000 7fb9a0c00640 1 -- 172.21.15.123:0/2224894400 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fb9a406d6b0 con 0x7fb98c0039b0 2024-04-25T19:03:54.562 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 msgr2=0x7fb9840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 0x7fb9840769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb9a414fab0 tx=0x7fb990008040 comp rx=0 tx=0).stop 2024-04-25T19:03:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 msgr2=0x7fb9a413edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c5e0 tx=0x7fb998001c10 comp rx=0 tx=0).stop 2024-04-25T19:03:54.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 shutdown_connections 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb9840744b0 0x7fb9840769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9a406d2a0 0x7fb9a4141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9a41f6780 0x7fb9a406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 --2- 172.21.15.123:0/2224894400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb98c0039b0 0x7fb9a413edd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.562+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 >> 172.21.15.123:0/2224894400 conn(0x7fb9a4073090 msgr2=0x7fb9a40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.563+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 shutdown_connections 2024-04-25T19:03:54.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.563+0000 7fb9a9e00640 1 -- 172.21.15.123:0/2224894400 wait complete. 2024-04-25T19:03:54.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:54.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.842+0000 7f1667e00640 1 Processor -- start 2024-04-25T19:03:54.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.842+0000 7f1667e00640 1 -- start start 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16680b0700 0x7f16680702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 0x7f166806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16680d1430 con 0x7f166815df10 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1668070f40 con 0x7f16680b0700 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1668161e90 con 0x7f166806d580 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1666000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16680b0700 0x7f16680702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1666000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16680b0700 0x7f16680702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56506/0 (socket says 172.21.15.123:56506) 2024-04-25T19:03:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1666000640 1 -- 172.21.15.123:0/1346737846 learned_addr learned my addr 172.21.15.123:0/1346737846 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1666000640 1 -- 172.21.15.123:0/1346737846 >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 msgr2=0x7f166806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.844+0000 7f1667400640 1 --2- 172.21.15.123:0/1346737846 >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 0x7f166806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.845+0000 7f1666000640 1 --2- 172.21.15.123:0/1346737846 >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 0x7f166806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.845+0000 7f1666000640 1 -- 172.21.15.123:0/1346737846 >> v1:172.21.15.146:6789/0 conn(0x7f166815df10 legacy=0x7f1668162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:54.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.845+0000 7f1666000640 1 -- 172.21.15.123:0/1346737846 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166806ded0 con 0x7f16680b0700 2024-04-25T19:03:54.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.845+0000 7f1667400640 1 --2- 172.21.15.123:0/1346737846 >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 0x7f166806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:54.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.846+0000 7f1666000640 1 --2- 172.21.15.123:0/1346737846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16680b0700 0x7f16680702f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f16680b1100 tx=0x7f165402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d43f8df1e0269e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.846+0000 7f1665600640 1 -- 172.21.15.123:0/1346737846 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654037070 con 0x7f16680b0700 2024-04-25T19:03:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.846+0000 7f1665600640 1 -- 172.21.15.123:0/1346737846 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1654032440 con 0x7f16680b0700 2024-04-25T19:03:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.846+0000 7f1665600640 1 -- 172.21.15.123:0/1346737846 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654032810 con 0x7f16680b0700 2024-04-25T19:03:54.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 -- 172.21.15.123:0/1346737846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16680b0700 msgr2=0x7f16680702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 --2- 172.21.15.123:0/1346737846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16680b0700 0x7f16680702f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f16680b1100 tx=0x7f165402a230 comp rx=0 tx=0).stop 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 -- 172.21.15.123:0/1346737846 shutdown_connections 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 --2- 172.21.15.123:0/1346737846 >> v2:172.21.15.146:3300/0 conn(0x7f166806d580 0x7f166806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 --2- 172.21.15.123:0/1346737846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16680b0700 0x7f16680702f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 -- 172.21.15.123:0/1346737846 >> 172.21.15.123:0/1346737846 conn(0x7f1668073090 msgr2=0x7f16680734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 -- 172.21.15.123:0/1346737846 shutdown_connections 2024-04-25T19:03:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.847+0000 7f1667e00640 1 -- 172.21.15.123:0/1346737846 wait complete. 2024-04-25T19:03:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.849+0000 7f1667e00640 1 Processor -- start 2024-04-25T19:03:54.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.849+0000 7f1667e00640 1 -- start start 2024-04-25T19:03:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166806d580 0x7f1668147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16680b0700 0x7f1668149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1668161b20 con 0x7f166815df10 2024-04-25T19:03:54.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1668070c60 con 0x7f166806d580 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16680cdfe0 con 0x7f16680b0700 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56508/0 (socket says 172.21.15.123:56508) 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 -- 172.21.15.123:0/1133580168 learned_addr learned my addr 172.21.15.123:0/1133580168 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1666000640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16680b0700 0x7f1668149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1666a00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166806d580 0x7f1668147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 -- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16680b0700 msgr2=0x7f1668149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16680b0700 0x7f1668149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 -- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166806d580 msgr2=0x7f1668147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166806d580 0x7f1668147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.851+0000 7f1667400640 1 -- 172.21.15.123:0/1133580168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1668147850 con 0x7f166815df10 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f1667400640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f166000cd80 tx=0x7f166000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000f680 con 0x7f166815df10 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166000f840 con 0x7f166815df10 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000fc30 con 0x7f166815df10 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166814a670 con 0x7f166815df10 2024-04-25T19:03:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.852+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1668147db0 con 0x7f166815df10 2024-04-25T19:03:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.853+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1668050740 con 0x7f166815df10 2024-04-25T19:03:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.854+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1660018400 con 0x7f166815df10 2024-04-25T19:03:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.855+0000 7f165ea00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 0x7f16480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.855+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1660096300 con 0x7f166815df10 2024-04-25T19:03:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.856+0000 7f1666a00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 0x7f16480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:54.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.856+0000 7f1666a00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 0x7f16480769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1658006fd0 tx=0x7f1658008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:54.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:54.860+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1660014030 con 0x7f166815df10 2024-04-25T19:03:55.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.063+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1668074190 con 0x7f166815df10 2024-04-25T19:03:55.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.064+0000 7f165ea00640 1 -- 172.21.15.123:0/1133580168 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f1660060720 con 0x7f166815df10 2024-04-25T19:03:55.066 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.066+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 msgr2=0x7f16480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.066+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 0x7f16480769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1658006fd0 tx=0x7f1658008040 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.066+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 msgr2=0x7f166815fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.066+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f166000cd80 tx=0x7f166000a650 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 shutdown_connections 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f16480744b0 0x7f16480769c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f166815df10 0x7f166815fe20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16680b0700 0x7f1668149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 --2- 172.21.15.123:0/1133580168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166806d580 0x7f1668147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 >> 172.21.15.123:0/1133580168 conn(0x7f1668073090 msgr2=0x7f16680b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:55.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 shutdown_connections 2024-04-25T19:03:55.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.067+0000 7f1667e00640 1 -- 172.21.15.123:0/1133580168 wait complete. 2024-04-25T19:03:55.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:55.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.367+0000 7fa879400640 1 Processor -- start 2024-04-25T19:03:55.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.367+0000 7fa879400640 1 -- start start 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa879400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 0x7fa8740cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa879400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8740cf500 0x7fa8740cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa879400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8740cfe50 con 0x7fa87406fcd0 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa879400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa8740707f0 con 0x7fa8740cf500 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa879400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa874070a20 con 0x7fa8740b08c0 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa873400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8740cf500 0x7fa8740cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa873400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8740cf500 0x7fa8740cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52000/0 (socket says 172.21.15.123:52000) 2024-04-25T19:03:55.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.370+0000 7fa873400640 1 -- 172.21.15.123:0/4218136908 learned_addr learned my addr 172.21.15.123:0/4218136908 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873e00640 1 --2- 172.21.15.123:0/4218136908 >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 0x7fa8740cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873400640 1 -- 172.21.15.123:0/4218136908 >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 msgr2=0x7fa8740cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873400640 1 --2- 172.21.15.123:0/4218136908 >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 0x7fa8740cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873400640 1 -- 172.21.15.123:0/4218136908 >> v1:172.21.15.146:6789/0 conn(0x7fa87406fcd0 legacy=0x7fa8740700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873400640 1 -- 172.21.15.123:0/4218136908 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87406c780 con 0x7fa8740cf500 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.371+0000 7fa873e00640 1 --2- 172.21.15.123:0/4218136908 >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 0x7fa8740cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.372+0000 7fa873400640 1 --2- 172.21.15.123:0/4218136908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cf500 0x7fa8740cf910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8740727f0 tx=0x7fa86002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fae489fd7b75bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.372+0000 7fa872a00640 1 -- 172.21.15.123:0/4218136908 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860037070 con 0x7fa8740cf500 2024-04-25T19:03:55.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.372+0000 7fa872a00640 1 -- 172.21.15.123:0/4218136908 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860032440 con 0x7fa8740cf500 2024-04-25T19:03:55.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.372+0000 7fa872a00640 1 -- 172.21.15.123:0/4218136908 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860032810 con 0x7fa8740cf500 2024-04-25T19:03:55.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 -- 172.21.15.123:0/4218136908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cf500 msgr2=0x7fa8740cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 --2- 172.21.15.123:0/4218136908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cf500 0x7fa8740cf910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8740727f0 tx=0x7fa86002a230 comp rx=0 tx=0).stop 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 -- 172.21.15.123:0/4218136908 shutdown_connections 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 --2- 172.21.15.123:0/4218136908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cf500 0x7fa8740cf910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 --2- 172.21.15.123:0/4218136908 >> v2:172.21.15.146:3300/0 conn(0x7fa8740b08c0 0x7fa8740cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 -- 172.21.15.123:0/4218136908 >> 172.21.15.123:0/4218136908 conn(0x7fa8740d19f0 msgr2=0x7fa8740d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 -- 172.21.15.123:0/4218136908 shutdown_connections 2024-04-25T19:03:55.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.373+0000 7fa879400640 1 -- 172.21.15.123:0/4218136908 wait complete. 2024-04-25T19:03:55.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.376+0000 7fa879400640 1 Processor -- start 2024-04-25T19:03:55.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.376+0000 7fa879400640 1 -- start start 2024-04-25T19:03:55.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.377+0000 7fa879400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 0x7fa87414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.377+0000 7fa879400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.377+0000 7fa879400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8740cf500 0x7fa87414dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.377+0000 7fa879400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa87406d260 con 0x7fa8740b08c0 2024-04-25T19:03:55.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa879400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa87406cd40 con 0x7fa87406fcd0 2024-04-25T19:03:55.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa879400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa87406d3e0 con 0x7fa8740cf500 2024-04-25T19:03:55.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa873400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa873400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52004/0 (socket says 172.21.15.123:52004) 2024-04-25T19:03:55.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa873400640 1 -- 172.21.15.123:0/2406132360 learned_addr learned my addr 172.21.15.123:0/2406132360 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:55.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa873e00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 0x7fa87414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.378+0000 7fa878a00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8740cf500 0x7fa87414dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 -- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8740cf500 msgr2=0x7fa87414dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8740cf500 0x7fa87414dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 -- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 msgr2=0x7fa87414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 0x7fa87414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 -- 172.21.15.123:0/2406132360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87414e400 con 0x7fa8740b08c0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873e00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 0x7fa87414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa873400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa8600090b0 tx=0x7fa8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860041070 con 0x7fa8740b08c0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86003c870 con 0x7fa8740b08c0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.379+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860037040 con 0x7fa8740b08c0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.380+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87414e6e0 con 0x7fa8740b08c0 2024-04-25T19:03:55.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.380+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741552a0 con 0x7fa8740b08c0 2024-04-25T19:03:55.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.381+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8740d0530 con 0x7fa8740b08c0 2024-04-25T19:03:55.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.382+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa860044070 con 0x7fa8740b08c0 2024-04-25T19:03:55.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.383+0000 7fa86be00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 0x7fa854076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.383+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa8600b6210 con 0x7fa8740b08c0 2024-04-25T19:03:55.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.383+0000 7fa873e00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 0x7fa854076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.384+0000 7fa873e00640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 0x7fa854076a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa874071530 tx=0x7fa864008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.388+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8740d0530 con 0x7fa8740b08c0 2024-04-25T19:03:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.590+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa87414e190 con 0x7fa8740b08c0 2024-04-25T19:03:55.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.591+0000 7fa86be00640 1 -- 172.21.15.123:0/2406132360 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fa8740cfe50 con 0x7fa8740b08c0 2024-04-25T19:03:55.592 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 msgr2=0x7fa854076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 0x7fa854076a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa874071530 tx=0x7fa864008040 comp rx=0 tx=0).stop 2024-04-25T19:03:55.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 msgr2=0x7fa87414d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa8600090b0 tx=0x7fa8600046c0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 shutdown_connections 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa854074500 0x7fa854076a10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8740cf500 0x7fa87414dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740b08c0 0x7fa87414d6b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 --2- 172.21.15.123:0/2406132360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa87406fcd0 0x7fa87414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 >> 172.21.15.123:0/2406132360 conn(0x7fa8740d19f0 msgr2=0x7fa8741604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.593+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 shutdown_connections 2024-04-25T19:03:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.594+0000 7fa879400640 1 -- 172.21.15.123:0/2406132360 wait complete. 2024-04-25T19:03:55.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.885+0000 7f61f0000640 1 Processor -- start 2024-04-25T19:03:55.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.885+0000 7f61f0000640 1 -- start start 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61f0000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f61e815dfc0 0x7f61e8162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61f0000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e815e3d0 con 0x7f61e806daa0 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61f0000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e80cc5f0 con 0x7f61e80cd140 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61f0000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f61e806deb0 con 0x7f61e815dfc0 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61ed600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f61e806daa0 0x7f61e80cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54448/0 (socket says 172.21.15.123:54448) 2024-04-25T19:03:55.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61ed600640 1 -- 172.21.15.123:0/2031853933 learned_addr learned my addr 172.21.15.123:0/2031853933 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.888+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088599664 0 0) 0x7f61dc001110 con 0x7f61e806daa0 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61d0003520 con 0x7f61e806daa0 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61ee000640 1 --2- 172.21.15.123:0/2031853933 >> v2:172.21.15.146:3300/0 conn(0x7f61e815dfc0 0x7f61e8162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 700623504 0 0) 0x7f61d0003520 con 0x7f61e806daa0 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 >> v2:172.21.15.146:3300/0 conn(0x7f61e815dfc0 msgr2=0x7f61e8162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 --2- 172.21.15.123:0/2031853933 >> v2:172.21.15.146:3300/0 conn(0x7f61e815dfc0 0x7f61e8162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 >> v1:172.21.15.146:6789/0 conn(0x7f61e80cd140 legacy=0x7f61e80af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e806ca00 con 0x7f61e806daa0 2024-04-25T19:03:55.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f61dc001840 con 0x7f61e806daa0 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 >> v1:172.21.15.123:6789/0 conn(0x7f61e806daa0 legacy=0x7f61e80cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 0x7f61d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61d0004800 0x7f61d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d0005270 0x7f61d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61e806deb0 con 0x7f61d00039b0 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000c200 con 0x7f61d0004800 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f61d000c400 con 0x7f61d0005270 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.889+0000 7f61ed600640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 0x7f61d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ee000640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d0005270 0x7f61d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ed600640 1 -- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d0005270 msgr2=0x7f61d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ed600640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d0005270 0x7f61d000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ed600640 1 -- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61d0004800 msgr2=0x7f61d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ed600640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61d0004800 0x7f61d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ed600640 1 -- 172.21.15.123:0/2031853933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d0003520 con 0x7f61d00039b0 2024-04-25T19:03:55.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.890+0000 7f61ee000640 1 -- 172.21.15.123:0/2031853933 reap_dead start 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.891+0000 7f61ed600640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 0x7f61d0003e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c490 tx=0x7f61d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d01599af34093fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.891+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00c0e0 con 0x7f61d00039b0 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.891+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc034070 con 0x7f61d00039b0 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.891+0000 7f61e7e00640 1 -- 172.21.15.123:0/2031853933 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc005d70 con 0x7f61d00039b0 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 -- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 msgr2=0x7f61d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 0x7f61d0003e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c490 tx=0x7f61d0005830 comp rx=0 tx=0).stop 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 -- 172.21.15.123:0/2031853933 shutdown_connections 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 --2- 172.21.15.123:0/2031853933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61d00039b0 0x7f61d0003e20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 -- 172.21.15.123:0/2031853933 >> 172.21.15.123:0/2031853933 conn(0x7f61e80ce390 msgr2=0x7f61e80ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 -- 172.21.15.123:0/2031853933 shutdown_connections 2024-04-25T19:03:55.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.892+0000 7f61f0000640 1 -- 172.21.15.123:0/2031853933 wait complete. 2024-04-25T19:03:55.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.895+0000 7f61f0000640 1 Processor -- start 2024-04-25T19:03:55.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.896+0000 7f61f0000640 1 -- start start 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d00039b0 0x7f61e815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e81f7650 0x7f61e81f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61e806cfc0 con 0x7f61e809edc0 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f61e806d610 con 0x7f61e81f7650 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61f0000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f61e806d2a0 con 0x7f61d00039b0 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52024/0 (socket says 172.21.15.123:52024) 2024-04-25T19:03:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 -- 172.21.15.123:0/3072822522 learned_addr learned my addr 172.21.15.123:0/3072822522 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ed600640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d00039b0 0x7f61e815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ee000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e81f7650 0x7f61e81f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 -- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d00039b0 msgr2=0x7f61e815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d00039b0 0x7f61e815a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 -- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e81f7650 msgr2=0x7f61e81f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.898+0000 7f61ecc00640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e81f7650 0x7f61e81f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:55.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61ecc00640 1 -- 172.21.15.123:0/3072822522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e81f7fa0 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61ecc00640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c460 tx=0x7f61d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40052d0 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4005490 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e81f3f30 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4012870 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.899+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e815ab40 con 0x7f61e809edc0 2024-04-25T19:03:55.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.900+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e8076a40 con 0x7f61e809edc0 2024-04-25T19:03:55.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.901+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61d4012a10 con 0x7f61e809edc0 2024-04-25T19:03:55.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.902+0000 7f61e5600640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 0x7f61c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:55.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.902+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f61d4095220 con 0x7f61e809edc0 2024-04-25T19:03:55.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.903+0000 7f61ed600640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 0x7f61c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:55.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.903+0000 7f61ed600640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 0x7f61c80769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61dc033000 tx=0x7f61dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:55.907+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61e81f7fa0 con 0x7f61e809edc0 2024-04-25T19:03:56.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.091+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f61e815a5f0 con 0x7f61e809edc0 2024-04-25T19:03:56.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.092+0000 7f61e5600640 1 -- 172.21.15.123:0/3072822522 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f61d405f5a0 con 0x7f61e809edc0 2024-04-25T19:03:56.094 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 msgr2=0x7f61c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 0x7f61c80769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61dc033000 tx=0x7f61dc02e000 comp rx=0 tx=0).stop 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 msgr2=0x7f61e81f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c460 tx=0x7f61d400c920 comp rx=0 tx=0).stop 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 shutdown_connections 2024-04-25T19:03:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61c80744b0 0x7f61c80769c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61e81f7650 0x7f61e81f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e809edc0 0x7f61e81f39f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 --2- 172.21.15.123:0/3072822522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61d00039b0 0x7f61e815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 >> 172.21.15.123:0/3072822522 conn(0x7f61e80ce390 msgr2=0x7f61e80b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 shutdown_connections 2024-04-25T19:03:56.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.095+0000 7f61f0000640 1 -- 172.21.15.123:0/3072822522 wait complete. 2024-04-25T19:03:56.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:03:56.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.389+0000 7efc52c00640 1 Processor -- start 2024-04-25T19:03:56.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.389+0000 7efc52c00640 1 -- start start 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc52c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efc4c15d7a0 0x7efc4c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc52c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4c06d750 con 0x7efc4c06d340 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc52c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4c0afe00 con 0x7efc4c156cd0 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc52c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efc4c15dbb0 con 0x7efc4c15d7a0 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc50c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efc4c06d340 0x7efc4c06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54464/0 (socket says 172.21.15.123:54464) 2024-04-25T19:03:56.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.391+0000 7efc50c00640 1 -- 172.21.15.123:0/2554800085 learned_addr learned my addr 172.21.15.123:0/2554800085 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:56.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581877776 0 0) 0x7efc44001110 con 0x7efc4c06d340 2024-04-25T19:03:56.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc43400640 1 --2- 172.21.15.123:0/2554800085 >> v2:172.21.15.146:3300/0 conn(0x7efc4c15d7a0 0x7efc4c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc30003520 con 0x7efc4c06d340 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 95632481 0 0) 0x7efc30003520 con 0x7efc4c06d340 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 >> v2:172.21.15.146:3300/0 conn(0x7efc4c15d7a0 msgr2=0x7efc4c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 --2- 172.21.15.123:0/2554800085 >> v2:172.21.15.146:3300/0 conn(0x7efc4c15d7a0 0x7efc4c06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 >> v1:172.21.15.146:6789/0 conn(0x7efc4c156cd0 legacy=0x7efc4c070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0b00a0 con 0x7efc4c06d340 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7efc44001840 con 0x7efc4c06d340 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 >> v1:172.21.15.123:6789/0 conn(0x7efc4c06d340 legacy=0x7efc4c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc300039b0 0x7efc30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 0x7efc30004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.392+0000 7efc4b400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc30005250 0x7efc3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c15dbb0 con 0x7efc30004800 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc3000c1e0 con 0x7efc300039b0 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc3000e5f0 con 0x7efc30005250 2024-04-25T19:03:56.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 0x7efc30004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc4be00640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc300039b0 0x7efc30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 -- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc30005250 msgr2=0x7efc3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc50c00640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc30005250 0x7efc3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc30005250 0x7efc3000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 -- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc300039b0 msgr2=0x7efc30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc300039b0 0x7efc30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc43400640 1 -- 172.21.15.123:0/2554800085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc30003520 con 0x7efc30004800 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc50c00640 1 -- 172.21.15.123:0/2554800085 reap_dead start 2024-04-25T19:03:56.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.393+0000 7efc4be00640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc300039b0 0x7efc30003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:56.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.394+0000 7efc43400640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 0x7efc30004c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc4c073100 tx=0x7efc30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6adebdc6e2bd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:56.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34009c10 con 0x7efc30004800 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc34036070 con 0x7efc30004800 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc4b400640 1 -- 172.21.15.123:0/2554800085 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34031830 con 0x7efc30004800 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 -- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 msgr2=0x7efc30004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 0x7efc30004c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc4c073100 tx=0x7efc30005830 comp rx=0 tx=0).stop 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 -- 172.21.15.123:0/2554800085 shutdown_connections 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 --2- 172.21.15.123:0/2554800085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30004800 0x7efc30004c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 -- 172.21.15.123:0/2554800085 >> 172.21.15.123:0/2554800085 conn(0x7efc4c0d1ae0 msgr2=0x7efc4c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 -- 172.21.15.123:0/2554800085 shutdown_connections 2024-04-25T19:03:56.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.395+0000 7efc52c00640 1 -- 172.21.15.123:0/2554800085 wait complete. 2024-04-25T19:03:56.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.398+0000 7efc52c00640 1 Processor -- start 2024-04-25T19:03:56.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.398+0000 7efc52c00640 1 -- start start 2024-04-25T19:03:56.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.399+0000 7efc52c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30004800 0x7efc4c1f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc52c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 0x7efc4c1ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc52c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc52c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0b0660 con 0x7efc4c15b4f0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc52c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c06cc80 con 0x7efc30004800 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc52c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c0b0940 con 0x7efc4c0b20f0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52046/0 (socket says 172.21.15.123:52046) 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 -- 172.21.15.123:0/3361639755 learned_addr learned my addr 172.21.15.123:0/3361639755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc4be00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30004800 0x7efc4c1f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc43400640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 0x7efc4c1ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 -- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 msgr2=0x7efc4c1ed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 0x7efc4c1ed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 -- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30004800 msgr2=0x7efc4c1f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30004800 0x7efc4c1f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc50c00640 1 -- 172.21.15.123:0/3361639755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c15be40 con 0x7efc4c15b4f0 2024-04-25T19:03:56.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.400+0000 7efc43400640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 0x7efc4c1ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:56.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc50c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e030 tx=0x7efc4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:56.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44007230 con 0x7efc4c15b4f0 2024-04-25T19:03:56.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc440073f0 con 0x7efc4c15b4f0 2024-04-25T19:03:56.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440047e0 con 0x7efc4c15b4f0 2024-04-25T19:03:56.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1ed810 con 0x7efc4c15b4f0 2024-04-25T19:03:56.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.401+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c0b1610 con 0x7efc4c15b4f0 2024-04-25T19:03:56.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.402+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c076a40 con 0x7efc4c15b4f0 2024-04-25T19:03:56.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.404+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc4401b060 con 0x7efc4c15b4f0 2024-04-25T19:03:56.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.405+0000 7efc48c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 0x7efc280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:56.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.405+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7efc44095490 con 0x7efc4c15b4f0 2024-04-25T19:03:56.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.405+0000 7efc4be00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 0x7efc280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:56.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.406+0000 7efc4be00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 0x7efc280769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc4c069e80 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:56.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.408+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc44022020 con 0x7efc4c15b4f0 2024-04-25T19:03:56.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.623+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc4c0b61d0 con 0x7efc4c15b4f0 2024-04-25T19:03:56.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.624+0000 7efc48c00640 1 -- 172.21.15.123:0/3361639755 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7efc4c15be40 con 0x7efc4c15b4f0 2024-04-25T19:03:56.625 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:03:56.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.626+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 msgr2=0x7efc280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.626+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 0x7efc280769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc4c069e80 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.626+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 msgr2=0x7efc4c15b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.626+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e030 tx=0x7efc4400dc00 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 shutdown_connections 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc280744b0 0x7efc280769c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c15b4f0 0x7efc4c15b900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc4c0b20f0 0x7efc4c1ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 --2- 172.21.15.123:0/3361639755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30004800 0x7efc4c1f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 >> 172.21.15.123:0/3361639755 conn(0x7efc4c0d1ae0 msgr2=0x7efc4c150410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 shutdown_connections 2024-04-25T19:03:56.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:56.627+0000 7efc52c00640 1 -- 172.21.15.123:0/3361639755 wait complete. 2024-04-25T19:03:56.653 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T19:03:56.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:56.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T19:03:56.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:03:56.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:03:56.753 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T19:03:56.820 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:56.821 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:56.874 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:56.874 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:56.921 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:03:56.921 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:03:57.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:03:57.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.293+0000 7f84a5800640 1 Processor -- start 2024-04-25T19:03:57.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.293+0000 7f84a5800640 1 -- start start 2024-04-25T19:03:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.295+0000 7f84a5800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84a015df00 0x7f84a006f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.295+0000 7f84a5800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84a006f780 0x7f84a00b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.295+0000 7f84a5800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a00ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.295+0000 7f84a5800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f84a015e310 con 0x7f84a015df00 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f84a5800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f84a00b5360 con 0x7f84a006f780 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f84a5800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84a00b6140 con 0x7f84a00b5d30 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84a015df00 0x7f84a006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84a015df00 0x7f84a006f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52054/0 (socket says 172.21.15.123:52054) 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 -- 172.21.15.123:0/1810341341 learned_addr learned my addr 172.21.15.123:0/1810341341 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 -- 172.21.15.123:0/1810341341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 msgr2=0x7f84a00ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:57.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 --2- 172.21.15.123:0/1810341341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a00ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 -- 172.21.15.123:0/1810341341 >> v2:172.21.15.146:3300/0 conn(0x7f84a006f780 msgr2=0x7f84a00b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 --2- 172.21.15.123:0/1810341341 >> v2:172.21.15.146:3300/0 conn(0x7f84a006f780 0x7f84a00b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.296+0000 7f849ea00640 1 -- 172.21.15.123:0/1810341341 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a00b4750 con 0x7f84a015df00 2024-04-25T19:03:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.297+0000 7f849ea00640 1 --2- 172.21.15.123:0/1810341341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a006f240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f84a0072780 tx=0x7f849002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c0a13b483270cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.298+0000 7f849e000640 1 -- 172.21.15.123:0/1810341341 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490037070 con 0x7f84a015df00 2024-04-25T19:03:57.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.298+0000 7f849e000640 1 -- 172.21.15.123:0/1810341341 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490032440 con 0x7f84a015df00 2024-04-25T19:03:57.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.298+0000 7f849e000640 1 -- 172.21.15.123:0/1810341341 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490032810 con 0x7f84a015df00 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.298+0000 7f84a5800640 1 -- 172.21.15.123:0/1810341341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 msgr2=0x7f84a006f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.298+0000 7f84a5800640 1 --2- 172.21.15.123:0/1810341341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a006f240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f84a0072780 tx=0x7f849002a160 comp rx=0 tx=0).stop 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 -- 172.21.15.123:0/1810341341 shutdown_connections 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 --2- 172.21.15.123:0/1810341341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a00ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 --2- 172.21.15.123:0/1810341341 >> v2:172.21.15.146:3300/0 conn(0x7f84a006f780 0x7f84a00b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 --2- 172.21.15.123:0/1810341341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a006f240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 -- 172.21.15.123:0/1810341341 >> 172.21.15.123:0/1810341341 conn(0x7f84a00d1ae0 msgr2=0x7f84a00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 -- 172.21.15.123:0/1810341341 shutdown_connections 2024-04-25T19:03:57.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.299+0000 7f84a5800640 1 -- 172.21.15.123:0/1810341341 wait complete. 2024-04-25T19:03:57.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.301+0000 7f84a5800640 1 Processor -- start 2024-04-25T19:03:57.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.301+0000 7f84a5800640 1 -- start start 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a006f780 0x7f84a00cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a0159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84a00b4ff0 con 0x7f84a015df00 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84a00b4d10 con 0x7f84a006f780 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f84a5800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84a00aed20 con 0x7f84a00b5d30 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52070/0 (socket says 172.21.15.123:52070) 2024-04-25T19:03:57.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849f400640 1 -- 172.21.15.123:0/2334470171 learned_addr learned my addr 172.21.15.123:0/2334470171 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f8497e00640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a0159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849f400640 1 -- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 msgr2=0x7f84a0159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849ea00640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a006f780 0x7f84a00cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.303+0000 7f849f400640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a0159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f849f400640 1 -- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a006f780 msgr2=0x7f84a00cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f849f400640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a006f780 0x7f84a00cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f849f400640 1 -- 172.21.15.123:0/2334470171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a00cb940 con 0x7f84a015df00 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f849f400640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f849800b9e0 tx=0x7f849800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498019070 con 0x7f84a015df00 2024-04-25T19:03:57.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849800cb20 con 0x7f84a015df00 2024-04-25T19:03:57.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a00cc840 con 0x7f84a015df00 2024-04-25T19:03:57.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498014440 con 0x7f84a015df00 2024-04-25T19:03:57.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.304+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a00cce00 con 0x7f84a015df00 2024-04-25T19:03:57.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.305+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a0158510 con 0x7f84a015df00 2024-04-25T19:03:57.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.307+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f849801c070 con 0x7f84a015df00 2024-04-25T19:03:57.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.308+0000 7f8496a00640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 0x7f847c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.308+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f8498095280 con 0x7f84a015df00 2024-04-25T19:03:57.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.308+0000 7f849ea00640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 0x7f847c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.309+0000 7f849ea00640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 0x7f847c0769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8490034000 tx=0x7f849002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.312+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f849805f690 con 0x7f84a015df00 2024-04-25T19:03:57.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.513+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a0076a40 con 0x7f84a015df00 2024-04-25T19:03:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.515+0000 7f8496a00640 1 -- 172.21.15.123:0/2334470171 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f849805ef90 con 0x7f84a015df00 2024-04-25T19:03:57.517 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:03:57.518 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 msgr2=0x7f847c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 0x7f847c0769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8490034000 tx=0x7f849002e000 comp rx=0 tx=0).stop 2024-04-25T19:03:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 msgr2=0x7f84a015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f849800b9e0 tx=0x7f849800bea0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 shutdown_connections 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f847c0744b0 0x7f847c0769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a015df00 0x7f84a015a3d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84a00b5d30 0x7f84a0159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 --2- 172.21.15.123:0/2334470171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a006f780 0x7f84a00cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.518+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 >> 172.21.15.123:0/2334470171 conn(0x7f84a00d1ae0 msgr2=0x7f84a0155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.519+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 shutdown_connections 2024-04-25T19:03:57.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.519+0000 7f84a5800640 1 -- 172.21.15.123:0/2334470171 wait complete. 2024-04-25T19:03:57.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:03:57.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:03:57.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.818+0000 7ff22f200640 1 Processor -- start 2024-04-25T19:03:57.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.818+0000 7ff22f200640 1 -- start start 2024-04-25T19:03:57.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.820+0000 7ff22f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.820+0000 7ff22f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff228070820 0x7ff2281609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.820+0000 7ff22f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2280cf300 con 0x7ff2280cdca0 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.820+0000 7ff22f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff2280cff10 con 0x7ff228070820 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.820+0000 7ff22f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2280cf5e0 con 0x7ff2280d03a0 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff228070820 0x7ff2281609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff228070820 0x7ff2281609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52086/0 (socket says 172.21.15.123:52086) 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 -- 172.21.15.123:0/3558477414 learned_addr learned my addr 172.21.15.123:0/3558477414 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:57.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff22d200640 1 --1- 172.21.15.123:0/3558477414 >> v1:172.21.15.123:6789/0 conn(0x7ff2280cdca0 0x7ff22815d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54468/0 (socket says 172.21.15.123:54468) 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227e00640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 -- 172.21.15.123:0/3558477414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 msgr2=0x7ff2280cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 -- 172.21.15.123:0/3558477414 >> v1:172.21.15.123:6789/0 conn(0x7ff2280cdca0 legacy=0x7ff22815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.821+0000 7ff227400640 1 -- 172.21.15.123:0/3558477414 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2280cf8c0 con 0x7ff228070820 2024-04-25T19:03:57.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.822+0000 7ff227400640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff228070820 0x7ff2281609e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff21000bb10 tx=0x7ff21002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3296cba0c9caf6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.823+0000 7ff226a00640 1 -- 172.21.15.123:0/3558477414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210036020 con 0x7ff228070820 2024-04-25T19:03:57.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.823+0000 7ff226a00640 1 -- 172.21.15.123:0/3558477414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210032820 con 0x7ff228070820 2024-04-25T19:03:57.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.823+0000 7ff226a00640 1 -- 172.21.15.123:0/3558477414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210032bf0 con 0x7ff228070820 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 -- 172.21.15.123:0/3558477414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff228070820 msgr2=0x7ff2281609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff228070820 0x7ff2281609e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff21000bb10 tx=0x7ff21002c920 comp rx=0 tx=0).stop 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 -- 172.21.15.123:0/3558477414 shutdown_connections 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff228070820 0x7ff2281609e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 --2- 172.21.15.123:0/3558477414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 -- 172.21.15.123:0/3558477414 >> 172.21.15.123:0/3558477414 conn(0x7ff2280ce390 msgr2=0x7ff2280ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 -- 172.21.15.123:0/3558477414 shutdown_connections 2024-04-25T19:03:57.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.824+0000 7ff22f200640 1 -- 172.21.15.123:0/3558477414 wait complete. 2024-04-25T19:03:57.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.827+0000 7ff22f200640 1 Processor -- start 2024-04-25T19:03:57.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.827+0000 7ff22f200640 1 -- start start 2024-04-25T19:03:57.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 0x7ff228157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff228160f20 con 0x7ff2280cdca0 2024-04-25T19:03:57.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff2280cc2c0 con 0x7ff228070820 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff22f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2281610a0 con 0x7ff2280d03a0 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff227400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff227400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52098/0 (socket says 172.21.15.123:52098) 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff227400640 1 -- 172.21.15.123:0/3315452374 learned_addr learned my addr 172.21.15.123:0/3315452374 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.829+0000 7ff227e00640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 0x7ff228157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 -- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 msgr2=0x7ff2280b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 -- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 msgr2=0x7ff228157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 0x7ff228157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 -- 172.21.15.123:0/3315452374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2280b4ba0 con 0x7ff2280cdca0 2024-04-25T19:03:57.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227e00640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 0x7ff228157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:57.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff227400640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff21002c960 tx=0x7ff210004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210042070 con 0x7ff2280cdca0 2024-04-25T19:03:57.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210031cb0 con 0x7ff2280cdca0 2024-04-25T19:03:57.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2281554d0 con 0x7ff2280cdca0 2024-04-25T19:03:57.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.830+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210036040 con 0x7ff2280cdca0 2024-04-25T19:03:57.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.831+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff228155a90 con 0x7ff2280cdca0 2024-04-25T19:03:57.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.832+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff228076a40 con 0x7ff2280cdca0 2024-04-25T19:03:57.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.833+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff210045070 con 0x7ff2280cdca0 2024-04-25T19:03:57.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.834+0000 7ff21fe00640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 0x7ff2040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:57.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.834+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7ff2100b61a0 con 0x7ff2280cdca0 2024-04-25T19:03:57.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.835+0000 7ff227e00640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 0x7ff2040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:57.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.835+0000 7ff227e00640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 0x7ff2040769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2280b0fa0 tx=0x7ff21800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:57.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:57.839+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff21002f070 con 0x7ff2280cdca0 2024-04-25T19:03:58.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.038+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff228069c60 con 0x7ff2280cdca0 2024-04-25T19:03:58.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.039+0000 7ff21fe00640 1 -- 172.21.15.123:0/3315452374 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7ff210080520 con 0x7ff2280cdca0 2024-04-25T19:03:58.041 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 msgr2=0x7ff2040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 0x7ff2040769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2280b0fa0 tx=0x7ff21800b040 comp rx=0 tx=0).stop 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 msgr2=0x7ff228150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff21002c960 tx=0x7ff210004730 comp rx=0 tx=0).stop 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 shutdown_connections 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff2040744e0 0x7ff2040769f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2280d03a0 0x7ff2280b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2280cdca0 0x7ff228150d60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 --2- 172.21.15.123:0/3315452374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff228070820 0x7ff228157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 >> 172.21.15.123:0/3315452374 conn(0x7ff2280ce390 msgr2=0x7ff2280ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:58.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 shutdown_connections 2024-04-25T19:03:58.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.042+0000 7ff22f200640 1 -- 172.21.15.123:0/3315452374 wait complete. 2024-04-25T19:03:58.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:03:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.346+0000 7f4646800640 1 Processor -- start 2024-04-25T19:03:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.346+0000 7f4646800640 1 -- start start 2024-04-25T19:03:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4646800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4640143040 0x7f464006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4646800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f46400cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4646800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464006ded0 con 0x7f46400cfff0 2024-04-25T19:03:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4646800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f46400d1200 con 0x7f4640143040 2024-04-25T19:03:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4646800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f46400d05f0 con 0x7f464006dac0 2024-04-25T19:03:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4645400640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f46400cfff0 0x7f464015dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:60204/0 (socket says 172.21.15.123:60204) 2024-04-25T19:03:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.349+0000 7f4645400640 1 -- 172.21.15.123:0/1971956952 learned_addr learned my addr 172.21.15.123:0/1971956952 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.350+0000 7f4645e00640 1 --2- 172.21.15.123:0/1971956952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f46400cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.350+0000 7f4644a00640 1 --2- 172.21.15.123:0/1971956952 >> v2:172.21.15.146:3300/0 conn(0x7f4640143040 0x7f464006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.350+0000 7f463fe00640 1 -- 172.21.15.123:0/1971956952 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517757859 0 0) 0x7f4634001110 con 0x7f46400cfff0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.350+0000 7f463fe00640 1 -- 172.21.15.123:0/1971956952 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4628003520 con 0x7f46400cfff0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.351+0000 7f4645e00640 1 -- 172.21.15.123:0/1971956952 >> v2:172.21.15.146:3300/0 conn(0x7f4640143040 msgr2=0x7f464006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.351+0000 7f4645e00640 1 --2- 172.21.15.123:0/1971956952 >> v2:172.21.15.146:3300/0 conn(0x7f4640143040 0x7f464006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.351+0000 7f4645e00640 1 -- 172.21.15.123:0/1971956952 >> v1:172.21.15.146:6789/0 conn(0x7f46400cfff0 legacy=0x7f464015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.351+0000 7f4645e00640 1 -- 172.21.15.123:0/1971956952 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46400d08d0 con 0x7f464006dac0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.352+0000 7f4645e00640 1 --2- 172.21.15.123:0/1971956952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f46400cd770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f463802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3df6b9d52c466d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.353+0000 7f463fe00640 1 -- 172.21.15.123:0/1971956952 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638007740 con 0x7f464006dac0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.353+0000 7f463fe00640 1 -- 172.21.15.123:0/1971956952 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380078e0 con 0x7f464006dac0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.353+0000 7f463fe00640 1 -- 172.21.15.123:0/1971956952 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638007cb0 con 0x7f464006dac0 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.353+0000 7f4646800640 1 -- 172.21.15.123:0/1971956952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 msgr2=0x7f46400cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.353+0000 7f4646800640 1 --2- 172.21.15.123:0/1971956952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f46400cd770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f463802a660 comp rx=0 tx=0).stop 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 -- 172.21.15.123:0/1971956952 shutdown_connections 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 --2- 172.21.15.123:0/1971956952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f46400cd770 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 --2- 172.21.15.123:0/1971956952 >> v2:172.21.15.146:3300/0 conn(0x7f4640143040 0x7f464006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 -- 172.21.15.123:0/1971956952 >> 172.21.15.123:0/1971956952 conn(0x7f4640073090 msgr2=0x7f46400734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 -- 172.21.15.123:0/1971956952 shutdown_connections 2024-04-25T19:03:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.354+0000 7f4646800640 1 -- 172.21.15.123:0/1971956952 wait complete. 2024-04-25T19:03:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.356+0000 7f4646800640 1 Processor -- start 2024-04-25T19:03:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.356+0000 7f4646800640 1 -- start start 2024-04-25T19:03:58.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f464015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 0x7f46400af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4640143450 con 0x7f4640143040 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f46400d0e90 con 0x7f46400cfff0 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4646800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f464013af50 con 0x7f464006dac0 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4645e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4645e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52106/0 (socket says 172.21.15.123:52106) 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4645e00640 1 -- 172.21.15.123:0/3348968611 learned_addr learned my addr 172.21.15.123:0/3348968611 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4644a00640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 0x7f46400af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.358+0000 7f4645400640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f464015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 -- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 msgr2=0x7f464015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f464015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 -- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 msgr2=0x7f46400af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 0x7f46400af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 -- 172.21.15.123:0/3348968611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46400b03b0 con 0x7f4640143040 2024-04-25T19:03:58.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645400640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f464015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:58.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4644a00640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 0x7f46400af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:58.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4645e00640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4638009eb0 tx=0x7f4638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803f070 con 0x7f4640143040 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46400b0690 con 0x7f4640143040 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.359+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638031bc0 con 0x7f4640143040 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.360+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638009450 con 0x7f4640143040 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.360+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46400b0c50 con 0x7f4640143040 2024-04-25T19:03:58.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.361+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46400b0120 con 0x7f4640143040 2024-04-25T19:03:58.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.362+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f463803b410 con 0x7f4640143040 2024-04-25T19:03:58.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.363+0000 7f463d600640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 0x7f46200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.363+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f46380b6ce0 con 0x7f4640143040 2024-04-25T19:03:58.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.364+0000 7f4645400640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 0x7f46200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.364+0000 7f4645400640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 0x7f46200769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.369+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4638080a20 con 0x7f4640143040 2024-04-25T19:03:58.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.570+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f46400b1900 con 0x7f4640143040 2024-04-25T19:03:58.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.571+0000 7f463d600640 1 -- 172.21.15.123:0/3348968611 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f46400b1900 con 0x7f4640143040 2024-04-25T19:03:58.573 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:58.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 msgr2=0x7f46200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 0x7f46200769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634009040 comp rx=0 tx=0).stop 2024-04-25T19:03:58.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 msgr2=0x7f46400cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4638009eb0 tx=0x7f4638004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 shutdown_connections 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46200744e0 0x7f46200769f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4640143040 0x7f46400cb8c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46400cfff0 0x7f46400af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.573+0000 7f4646800640 1 --2- 172.21.15.123:0/3348968611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f464006dac0 0x7f464015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.574+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 >> 172.21.15.123:0/3348968611 conn(0x7f4640073090 msgr2=0x7f46400b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.574+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 shutdown_connections 2024-04-25T19:03:58.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.574+0000 7f4646800640 1 -- 172.21.15.123:0/3348968611 wait complete. 2024-04-25T19:03:58.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:03:58.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.873+0000 7f94fac00640 1 Processor -- start 2024-04-25T19:03:58.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.873+0000 7f94fac00640 1 -- start start 2024-04-25T19:03:58.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.874+0000 7f94fac00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 0x7f94f406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.874+0000 7f94fac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94f41647a0 0x7f94f406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.874+0000 7f94fac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40d4fb0 con 0x7f94f40b46b0 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.874+0000 7f94fac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f94f40d5570 con 0x7f94f41647a0 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.874+0000 7f94fac00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f94f406c7a0 con 0x7f94f4164390 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94f41647a0 0x7f94f406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94f41647a0 0x7f94f406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52116/0 (socket says 172.21.15.123:52116) 2024-04-25T19:03:58.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 -- 172.21.15.123:0/2670744452 learned_addr learned my addr 172.21.15.123:0/2670744452 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:58.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f3400640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 0x7f94f406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 -- 172.21.15.123:0/2670744452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 msgr2=0x7f94f406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:58.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 0x7f94f406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 -- 172.21.15.123:0/2670744452 >> v1:172.21.15.123:6789/0 conn(0x7f94f40b46b0 legacy=0x7f94f4163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f8c00640 1 -- 172.21.15.123:0/2670744452 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40d5290 con 0x7f94f41647a0 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f2a00640 1 -- 172.21.15.123:0/2670744452 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11206726 0 0) 0x7f94f40d4fb0 con 0x7f94f40b46b0 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.875+0000 7f94f3400640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 0x7f94f406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.876+0000 7f94f8c00640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41647a0 0x7f94f406d170 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009870 tx=0x7f94ec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95818ab2016d925a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.877+0000 7f94f2a00640 1 -- 172.21.15.123:0/2670744452 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec037070 con 0x7f94f41647a0 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.877+0000 7f94f2a00640 1 -- 172.21.15.123:0/2670744452 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec032440 con 0x7f94f41647a0 2024-04-25T19:03:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.877+0000 7f94f2a00640 1 -- 172.21.15.123:0/2670744452 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec032810 con 0x7f94f41647a0 2024-04-25T19:03:58.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.877+0000 7f94fac00640 1 -- 172.21.15.123:0/2670744452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41647a0 msgr2=0x7f94f406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.877+0000 7f94fac00640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41647a0 0x7f94f406d170 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009870 tx=0x7f94ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 -- 172.21.15.123:0/2670744452 shutdown_connections 2024-04-25T19:03:58.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41647a0 0x7f94f406d170 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 --2- 172.21.15.123:0/2670744452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f4164390 0x7f94f406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 -- 172.21.15.123:0/2670744452 >> 172.21.15.123:0/2670744452 conn(0x7f94f40d6650 msgr2=0x7f94f40b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:58.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 -- 172.21.15.123:0/2670744452 shutdown_connections 2024-04-25T19:03:58.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.878+0000 7f94fac00640 1 -- 172.21.15.123:0/2670744452 wait complete. 2024-04-25T19:03:58.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.880+0000 7f94fac00640 1 Processor -- start 2024-04-25T19:03:58.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.880+0000 7f94fac00640 1 -- start start 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 0x7f94f415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f41647a0 0x7f94f40c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41603f0 con 0x7f94f4164390 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f94f40b4f70 con 0x7f94f40b46b0 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94fac00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f94f41f7e40 con 0x7f94f41647a0 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52122/0 (socket says 172.21.15.123:52122) 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 -- 172.21.15.123:0/4004670888 learned_addr learned my addr 172.21.15.123:0/4004670888 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3e00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 0x7f94f415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 -- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f41647a0 msgr2=0x7f94f40c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f41647a0 0x7f94f40c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 -- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 msgr2=0x7f94f415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 0x7f94f415cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3400640 1 -- 172.21.15.123:0/4004670888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f415a280 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.882+0000 7f94f3e00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 0x7f94f415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94f3400640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c7f0 tx=0x7f94e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0015070 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e0007820 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f4161960 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0007bd0 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.883+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f4161f20 con 0x7f94f4164390 2024-04-25T19:03:58.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.885+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f4076a40 con 0x7f94f4164390 2024-04-25T19:03:58.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.886+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f94e0018070 con 0x7f94f4164390 2024-04-25T19:03:58.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.887+0000 7f94ebe00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 0x7f94d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:58.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.887+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f94e0095770 con 0x7f94f4164390 2024-04-25T19:03:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.888+0000 7f94f3e00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 0x7f94d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:58.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.888+0000 7f94f3e00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 0x7f94d40769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94e40012e0 tx=0x7f94e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:58.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:58.892+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94e005fb90 con 0x7f94f4164390 2024-04-25T19:03:59.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.098+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94f415a010 con 0x7f94f4164390 2024-04-25T19:03:59.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.099+0000 7f94ebe00640 1 -- 172.21.15.123:0/4004670888 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f94e005f490 con 0x7f94f4164390 2024-04-25T19:03:59.102 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:03:59.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 msgr2=0x7f94d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 0x7f94d40769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94e40012e0 tx=0x7f94e4009040 comp rx=0 tx=0).stop 2024-04-25T19:03:59.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 msgr2=0x7f94f41f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c7f0 tx=0x7f94e000ccb0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 shutdown_connections 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94d40744b0 0x7f94d40769c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94f41647a0 0x7f94f40c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f4164390 0x7f94f41f6b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 --2- 172.21.15.123:0/4004670888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94f40b46b0 0x7f94f415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 >> 172.21.15.123:0/4004670888 conn(0x7f94f40d6650 msgr2=0x7f94f40c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 shutdown_connections 2024-04-25T19:03:59.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.102+0000 7f94fac00640 1 -- 172.21.15.123:0/4004670888 wait complete. 2024-04-25T19:03:59.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:03:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.400+0000 7f851d000640 1 Processor -- start 2024-04-25T19:03:59.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.400+0000 7f851d000640 1 -- start start 2024-04-25T19:03:59.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f851d000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f851815df10 0x7f85180702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f851d000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f8518162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f851d000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85180d1430 con 0x7f851806d580 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f851d000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8518070f40 con 0x7f851815df10 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f851d000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8518161e90 con 0x7f85180b0700 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8517e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f851806d580 0x7f851806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54484/0 (socket says 172.21.15.123:54484) 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8517e00640 1 -- 172.21.15.123:0/3175309403 learned_addr learned my addr 172.21.15.123:0/3175309403 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516a00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f8518162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044111613 0 0) 0x7f85180d1430 con 0x7f851806d580 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8517400640 1 --2- 172.21.15.123:0/3175309403 >> v2:172.21.15.146:3300/0 conn(0x7f851815df10 0x7f85180702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84fc003520 con 0x7f851806d580 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3073298627 0 0) 0x7f84fc003520 con 0x7f851806d580 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 msgr2=0x7f8518162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f8518162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 >> v2:172.21.15.146:3300/0 conn(0x7f851815df10 msgr2=0x7f85180702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 --2- 172.21.15.123:0/3175309403 >> v2:172.21.15.146:3300/0 conn(0x7f851815df10 0x7f85180702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8518070980 con 0x7f851806d580 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f85100019f0 con 0x7f851806d580 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 >> v1:172.21.15.123:6789/0 conn(0x7f851806d580 legacy=0x7f851806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc0039b0 0x7f84fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.403+0000 7f8516000640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f84fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8516000640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84fc004d40 0x7f84fc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8518161e90 con 0x7f84fc004d40 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc00ba70 con 0x7f84fc0039b0 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84fc00bbf0 con 0x7f85180b0700 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84fc004d40 0x7f84fc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8516a00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f84fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517400640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc0039b0 0x7f84fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 -- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 msgr2=0x7f84fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85180b0700 0x7f84fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 -- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc0039b0 msgr2=0x7f84fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc0039b0 0x7f84fc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517e00640 1 -- 172.21.15.123:0/3175309403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc003520 con 0x7f84fc004d40 2024-04-25T19:03:59.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.404+0000 7f8517400640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc0039b0 0x7f84fc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:59.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.405+0000 7f8517e00640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84fc004d40 0x7f84fc005150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f851000a8a0 tx=0x7f84fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9fa2fb408c2f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000a4c0 con 0x7f84fc004d40 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851002f070 con 0x7f84fc004d40 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f8516000640 1 -- 172.21.15.123:0/3175309403 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851002bd30 con 0x7f84fc004d40 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f851d000640 1 -- 172.21.15.123:0/3175309403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84fc004d40 msgr2=0x7f84fc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f851d000640 1 --2- 172.21.15.123:0/3175309403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84fc004d40 0x7f84fc005150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f851000a8a0 tx=0x7f84fc005830 comp rx=0 tx=0).stop 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f8517e00640 1 -- 172.21.15.123:0/3175309403 reap_dead start 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f851d000640 1 -- 172.21.15.123:0/3175309403 shutdown_connections 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.406+0000 7f851d000640 1 -- 172.21.15.123:0/3175309403 >> 172.21.15.123:0/3175309403 conn(0x7f8518073090 msgr2=0x7f85180734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.407+0000 7f851d000640 1 -- 172.21.15.123:0/3175309403 shutdown_connections 2024-04-25T19:03:59.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.407+0000 7f851d000640 1 -- 172.21.15.123:0/3175309403 wait complete. 2024-04-25T19:03:59.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.409+0000 7f851d000640 1 Processor -- start 2024-04-25T19:03:59.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.409+0000 7f851d000640 1 -- start start 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc004d40 0x7f851814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f851815a800 0x7f85180c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8518162320 con 0x7f85180b21b0 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8518161b20 con 0x7f84fc004d40 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f851d000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f85181624a0 con 0x7f851815a800 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52142/0 (socket says 172.21.15.123:52142) 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 -- 172.21.15.123:0/2552614543 learned_addr learned my addr 172.21.15.123:0/2552614543 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8516a00640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f851815a800 0x7f85180c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 -- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f851815a800 msgr2=0x7f85180c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f851815a800 0x7f85180c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 -- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc004d40 msgr2=0x7f851814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517400640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc004d40 0x7f851814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.411+0000 7f8517e00640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc004d40 0x7f851814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f8517e00640 1 -- 172.21.15.123:0/2552614543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85180b2b00 con 0x7f85180b21b0 2024-04-25T19:03:59.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f8517e00640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f851000a8a0 tx=0x7f8510001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510044070 con 0x7f85180b21b0 2024-04-25T19:03:59.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8510034670 con 0x7f85180b21b0 2024-04-25T19:03:59.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000a4c0 con 0x7f85180b21b0 2024-04-25T19:03:59.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851815ad30 con 0x7f85180b21b0 2024-04-25T19:03:59.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.412+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851815b2f0 con 0x7f85180b21b0 2024-04-25T19:03:59.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.413+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851806de50 con 0x7f85180b21b0 2024-04-25T19:03:59.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.415+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8510034a20 con 0x7f85180b21b0 2024-04-25T19:03:59.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.416+0000 7f850f400640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 0x7f84f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.416+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f85100b5710 con 0x7f85180b21b0 2024-04-25T19:03:59.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.417+0000 7f8517400640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 0x7f84f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.417+0000 7f8517400640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 0x7f84f80769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.422+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f851806de50 con 0x7f85180b21b0 2024-04-25T19:03:59.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.613+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8518050740 con 0x7f85180b21b0 2024-04-25T19:03:59.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.614+0000 7f850f400640 1 -- 172.21.15.123:0/2552614543 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f85180b2b00 con 0x7f85180b21b0 2024-04-25T19:03:59.617 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:03:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.616+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 msgr2=0x7f84f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.616+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 0x7f84f80769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-04-25T19:03:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 msgr2=0x7f85180b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f851000a8a0 tx=0x7f8510001dc0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 shutdown_connections 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f84f80744b0 0x7f84f80769c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85180b21b0 0x7f85180b25c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f851815a800 0x7f85180c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 --2- 172.21.15.123:0/2552614543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84fc004d40 0x7f851814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 >> 172.21.15.123:0/2552614543 conn(0x7f8518073090 msgr2=0x7f851815e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 shutdown_connections 2024-04-25T19:03:59.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.617+0000 7f851d000640 1 -- 172.21.15.123:0/2552614543 wait complete. 2024-04-25T19:03:59.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:03:59.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.913+0000 7fc28e400640 1 Processor -- start 2024-04-25T19:03:59.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.913+0000 7fc28e400640 1 -- start start 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc28e400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc288161b10 0x7fc288161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc28e400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc288154950 0x7fc288154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc28e400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc288071600 con 0x7fc2880d1120 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc28e400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc2880d1530 con 0x7fc288161b10 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc28e400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc288162460 con 0x7fc288154950 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc286a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc288161b10 0x7fc288161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc287400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc2880d1120 0x7fc288070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54498/0 (socket says 172.21.15.123:54498) 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.915+0000 7fc287400640 1 -- 172.21.15.123:0/4023253243 learned_addr learned my addr 172.21.15.123:0/4023253243 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc287e00640 1 --2- 172.21.15.123:0/4023253243 >> v2:172.21.15.146:3300/0 conn(0x7fc288154950 0x7fc288154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc286a00640 1 -- 172.21.15.123:0/4023253243 >> v2:172.21.15.146:3300/0 conn(0x7fc288154950 msgr2=0x7fc288154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:59.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc286a00640 1 --2- 172.21.15.123:0/4023253243 >> v2:172.21.15.146:3300/0 conn(0x7fc288154950 0x7fc288154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc286a00640 1 -- 172.21.15.123:0/4023253243 >> v1:172.21.15.123:6789/0 conn(0x7fc2880d1120 legacy=0x7fc288070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc286a00640 1 -- 172.21.15.123:0/4023253243 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2880aec30 con 0x7fc288161b10 2024-04-25T19:03:59.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.916+0000 7fc286000640 1 -- 172.21.15.123:0/4023253243 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323945109 0 0) 0x7fc288071600 con 0x7fc2880d1120 2024-04-25T19:03:59.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.917+0000 7fc286a00640 1 --2- 172.21.15.123:0/4023253243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288161b10 0x7fc288161f20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2880730e0 tx=0x7fc27802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1239b88e4e20990a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.917+0000 7fc286000640 1 -- 172.21.15.123:0/4023253243 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278007740 con 0x7fc288161b10 2024-04-25T19:03:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc286000640 1 -- 172.21.15.123:0/4023253243 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2780078e0 con 0x7fc288161b10 2024-04-25T19:03:59.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc286000640 1 -- 172.21.15.123:0/4023253243 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278007cb0 con 0x7fc288161b10 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 -- 172.21.15.123:0/4023253243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288161b10 msgr2=0x7fc288161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 --2- 172.21.15.123:0/4023253243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288161b10 0x7fc288161f20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2880730e0 tx=0x7fc27802a560 comp rx=0 tx=0).stop 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 -- 172.21.15.123:0/4023253243 shutdown_connections 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 --2- 172.21.15.123:0/4023253243 >> v2:172.21.15.146:3300/0 conn(0x7fc288154950 0x7fc288154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 --2- 172.21.15.123:0/4023253243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288161b10 0x7fc288161f20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 -- 172.21.15.123:0/4023253243 >> 172.21.15.123:0/4023253243 conn(0x7fc2880d1ae0 msgr2=0x7fc2880d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.918+0000 7fc28e400640 1 -- 172.21.15.123:0/4023253243 shutdown_connections 2024-04-25T19:03:59.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.919+0000 7fc28e400640 1 -- 172.21.15.123:0/4023253243 wait complete. 2024-04-25T19:03:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.921+0000 7fc28e400640 1 Processor -- start 2024-04-25T19:03:59.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.921+0000 7fc28e400640 1 -- start start 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2880d1120 0x7fc2881e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc288161b10 0x7fc288155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2880af4d0 con 0x7fc288154950 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2880af1f0 con 0x7fc288161b10 2024-04-25T19:03:59.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.922+0000 7fc28e400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc2881552a0 con 0x7fc2880d1120 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52166/0 (socket says 172.21.15.123:52166) 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 -- 172.21.15.123:0/2951211106 learned_addr learned my addr 172.21.15.123:0/2951211106 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc287400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2880d1120 0x7fc2881e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 -- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2880d1120 msgr2=0x7fc2881e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2880d1120 0x7fc2881e83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc287e00640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc288161b10 0x7fc288155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 -- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc288161b10 msgr2=0x7fc288155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc288161b10 0x7fc288155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.923+0000 7fc286a00640 1 -- 172.21.15.123:0/2951211106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc288156580 con 0x7fc288154950 2024-04-25T19:03:59.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc286a00640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc27802ab50 tx=0x7fc278004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27803f070 con 0x7fc288154950 2024-04-25T19:03:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc278031c80 con 0x7fc288154950 2024-04-25T19:03:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278044660 con 0x7fc288154950 2024-04-25T19:03:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc288156860 con 0x7fc288154950 2024-04-25T19:03:59.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.924+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc288156e20 con 0x7fc288154950 2024-04-25T19:03:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.925+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2881f6e20 con 0x7fc288154950 2024-04-25T19:03:59.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.926+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc278044800 con 0x7fc288154950 2024-04-25T19:03:59.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.927+0000 7fc27f400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 0x7fc2640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:59.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.928+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc2780b6e30 con 0x7fc288154950 2024-04-25T19:03:59.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.928+0000 7fc287400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 0x7fc2640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:59.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.928+0000 7fc287400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 0x7fc2640769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc270000f60 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:59.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:03:59.932+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc278080b60 con 0x7fc288154950 2024-04-25T19:04:00.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.151+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc288156310 con 0x7fc288154950 2024-04-25T19:04:00.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.152+0000 7fc27f400640 1 -- 172.21.15.123:0/2951211106 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fc278084e60 con 0x7fc288154950 2024-04-25T19:04:00.154 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.155+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 msgr2=0x7fc2640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.155+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 0x7fc2640769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc270000f60 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.155+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 msgr2=0x7fc288155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.155+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc27802ab50 tx=0x7fc278004a40 comp rx=0 tx=0).stop 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 shutdown_connections 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc2640744b0 0x7fc2640769c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc288161b10 0x7fc288155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc288154950 0x7fc288155800 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 --2- 172.21.15.123:0/2951211106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2880d1120 0x7fc2881e83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 >> 172.21.15.123:0/2951211106 conn(0x7fc2880d1ae0 msgr2=0x7fc2880b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:00.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 shutdown_connections 2024-04-25T19:04:00.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.156+0000 7fc28e400640 1 -- 172.21.15.123:0/2951211106 wait complete. 2024-04-25T19:04:00.182 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T19:04:00.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:00.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T19:04:00.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:04:00.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:00.272 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T19:04:00.320 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:04:00.320 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:04:00.381 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:04:00.381 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:04:00.436 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:04:00.436 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:04:00.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:00.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.770+0000 7f9c5e000640 1 Processor -- start 2024-04-25T19:04:00.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.770+0000 7f9c5e000640 1 -- start start 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c5e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c5806d510 0x7f9c58167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c5e000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c580afe00 con 0x7f9c5815d7a0 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c5e000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c580b00a0 con 0x7f9c58156cd0 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c5e000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5815dbb0 con 0x7f9c5806d510 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c57e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c5806d510 0x7f9c58167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c57e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c5806d510 0x7f9c58167510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52174/0 (socket says 172.21.15.123:52174) 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c57e00640 1 -- 172.21.15.123:0/2487609765 learned_addr learned my addr 172.21.15.123:0/2487609765 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:00.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c56a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9c5815d7a0 0x7f9c5806ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54508/0 (socket says 172.21.15.123:54508) 2024-04-25T19:04:00.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.772+0000 7f9c57e00640 1 -- 172.21.15.123:0/2487609765 >> v1:172.21.15.146:6789/0 conn(0x7f9c58156cd0 legacy=0x7f9c58070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:00.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.773+0000 7f9c57e00640 1 -- 172.21.15.123:0/2487609765 >> v1:172.21.15.123:6789/0 conn(0x7f9c5815d7a0 legacy=0x7f9c5806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.773+0000 7f9c57e00640 1 -- 172.21.15.123:0/2487609765 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c580b0380 con 0x7f9c5806d510 2024-04-25T19:04:00.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.774+0000 7f9c57e00640 1 --2- 172.21.15.123:0/2487609765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5806d510 0x7f9c58167510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009870 tx=0x7f9c5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e2d4f9fc8fd21e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:00.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.774+0000 7f9c56000640 1 -- 172.21.15.123:0/2487609765 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50037070 con 0x7f9c5806d510 2024-04-25T19:04:00.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.774+0000 7f9c56000640 1 -- 172.21.15.123:0/2487609765 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c50032440 con 0x7f9c5806d510 2024-04-25T19:04:00.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.774+0000 7f9c56000640 1 -- 172.21.15.123:0/2487609765 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50032810 con 0x7f9c5806d510 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 -- 172.21.15.123:0/2487609765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5806d510 msgr2=0x7f9c58167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 --2- 172.21.15.123:0/2487609765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5806d510 0x7f9c58167510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009870 tx=0x7f9c5002a210 comp rx=0 tx=0).stop 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 -- 172.21.15.123:0/2487609765 shutdown_connections 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 --2- 172.21.15.123:0/2487609765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5806d510 0x7f9c58167510 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 -- 172.21.15.123:0/2487609765 >> 172.21.15.123:0/2487609765 conn(0x7f9c580d1ae0 msgr2=0x7f9c580d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 -- 172.21.15.123:0/2487609765 shutdown_connections 2024-04-25T19:04:00.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.775+0000 7f9c5e000640 1 -- 172.21.15.123:0/2487609765 wait complete. 2024-04-25T19:04:00.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.777+0000 7f9c5e000640 1 Processor -- start 2024-04-25T19:04:00.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.777+0000 7f9c5e000640 1 -- start start 2024-04-25T19:04:00.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.778+0000 7f9c5e000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c58156cd0 0x7f9c5814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c5e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c5e000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c58132310 0x7f9c58133db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c5e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580b0940 con 0x7f9c5815d7a0 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c5e000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58070250 con 0x7f9c58132310 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c5e000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c580b0e60 con 0x7f9c58156cd0 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52188/0 (socket says 172.21.15.123:52188) 2024-04-25T19:04:00.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 -- 172.21.15.123:0/1003143841 learned_addr learned my addr 172.21.15.123:0/1003143841 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:00.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 -- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c58156cd0 msgr2=0x7f9c5814aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c58156cd0 0x7f9c5814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 -- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c58132310 msgr2=0x7f9c58133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c57e00640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c58132310 0x7f9c58133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c58132310 0x7f9c58133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.779+0000 7f9c56a00640 1 -- 172.21.15.123:0/1003143841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5814b420 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c56a00640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400c3c0 tx=0x7f9c4400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440074e0 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c44007680 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c58132b70 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004a00 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.780+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c58133130 con 0x7f9c5815d7a0 2024-04-25T19:04:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.781+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c581328c0 con 0x7f9c5815d7a0 2024-04-25T19:04:00.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.782+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c4401a020 con 0x7f9c5815d7a0 2024-04-25T19:04:00.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.783+0000 7f9c4f400640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 0x7f9c380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:00.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.784+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f9c44094d00 con 0x7f9c5815d7a0 2024-04-25T19:04:00.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.784+0000 7f9c57400640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 0x7f9c380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:00.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.784+0000 7f9c57400640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 0x7f9c380769f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c58074640 tx=0x7f9c48009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:00.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.788+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c5814b420 con 0x7f9c5815d7a0 2024-04-25T19:04:00.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.972+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c58076a40 con 0x7f9c5815d7a0 2024-04-25T19:04:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.974+0000 7f9c4f400640 1 -- 172.21.15.123:0/1003143841 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f9c4405f080 con 0x7f9c5815d7a0 2024-04-25T19:04:00.976 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:00.976 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 msgr2=0x7f9c380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 0x7f9c380769f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c58074640 tx=0x7f9c48009230 comp rx=0 tx=0).stop 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 msgr2=0x7f9c58131dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400c3c0 tx=0x7f9c4400c7c0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 shutdown_connections 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9c380744e0 0x7f9c380769f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c58132310 0x7f9c58133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c5815d7a0 0x7f9c58131dd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 --2- 172.21.15.123:0/1003143841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c58156cd0 0x7f9c5814aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:00.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 >> 172.21.15.123:0/1003143841 conn(0x7f9c580d1ae0 msgr2=0x7f9c5806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:00.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 shutdown_connections 2024-04-25T19:04:00.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:00.977+0000 7f9c5e000640 1 -- 172.21.15.123:0/1003143841 wait complete. 2024-04-25T19:04:01.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:04:01.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:04:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.269+0000 7fbdfb200640 1 Processor -- start 2024-04-25T19:04:01.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.270+0000 7fbdfb200640 1 -- start start 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 0x7fbdfc1645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdfb200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbdfc06dc90 0x7fbdfc160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdfb200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdfc163b20 con 0x7fbdfc160530 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdfb200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbdfc164150 con 0x7fbdfc06dc90 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdfb200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc161060 con 0x7fbdfc0b0ac0 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdf9400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbdfc160530 0x7fbdfc06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54524/0 (socket says 172.21.15.123:54524) 2024-04-25T19:04:01.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.272+0000 7fbdf9400640 1 -- 172.21.15.123:0/2788060434 learned_addr learned my addr 172.21.15.123:0/2788060434 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 0x7fbdfc1645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdfa800640 1 --2- 172.21.15.123:0/2788060434 >> v2:172.21.15.146:3300/0 conn(0x7fbdfc06dc90 0x7fbdfc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 926856460 0 0) 0x7fbdfc163b20 con 0x7fbdfc160530 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbde0003520 con 0x7fbdfc160530 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3905654686 0 0) 0x7fbde0003520 con 0x7fbdfc160530 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 msgr2=0x7fbdfc1645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 0x7fbdfc1645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 >> v2:172.21.15.146:3300/0 conn(0x7fbdfc06dc90 msgr2=0x7fbdfc160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 --2- 172.21.15.123:0/2788060434 >> v2:172.21.15.146:3300/0 conn(0x7fbdfc06dc90 0x7fbdfc160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc161340 con 0x7fbdfc160530 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fbdec0019f0 con 0x7fbdfc160530 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 >> v1:172.21.15.123:6789/0 conn(0x7fbdfc160530 legacy=0x7fbdfc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde00039b0 0x7fbde0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.273+0000 7fbdf8a00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 0x7fbde0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf8a00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbde0004d40 0x7fbde0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc164150 con 0x7fbde00039b0 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc161060 con 0x7fbde0004d40 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbde000ba40 con 0x7fbdfc0b0ac0 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde00039b0 0x7fbde0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 -- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 msgr2=0x7fbde0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdfc0b0ac0 0x7fbde0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 -- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbde0004d40 msgr2=0x7fbde0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbde0004d40 0x7fbde0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.274+0000 7fbdf9e00640 1 -- 172.21.15.123:0/2788060434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde0003520 con 0x7fbde00039b0 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.275+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde00039b0 0x7fbde0003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009ac0 tx=0x7fbde0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa7f52f4707af6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.275+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4036070 con 0x7fbde00039b0 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.275+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde4031400 con 0x7fbde00039b0 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.275+0000 7fbdf8a00640 1 -- 172.21.15.123:0/2788060434 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4009690 con 0x7fbde00039b0 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 -- 172.21.15.123:0/2788060434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde00039b0 msgr2=0x7fbde0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2788060434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde00039b0 0x7fbde0003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009ac0 tx=0x7fbde0005830 comp rx=0 tx=0).stop 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfa800640 1 -- 172.21.15.123:0/2788060434 reap_dead start 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 -- 172.21.15.123:0/2788060434 shutdown_connections 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 -- 172.21.15.123:0/2788060434 >> 172.21.15.123:0/2788060434 conn(0x7fbdfc073090 msgr2=0x7fbdfc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 -- 172.21.15.123:0/2788060434 shutdown_connections 2024-04-25T19:04:01.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.276+0000 7fbdfb200640 1 -- 172.21.15.123:0/2788060434 wait complete. 2024-04-25T19:04:01.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.279+0000 7fbdfb200640 1 Processor -- start 2024-04-25T19:04:01.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.279+0000 7fbdfb200640 1 -- start start 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.280+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 0x7fbdfc158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.280+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.280+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc0d0920 0x7fbdfc163010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdfb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc1645e0 con 0x7fbdfc142960 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdfb200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc070b20 con 0x7fbdfc0d0920 2024-04-25T19:04:01.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdfb200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc164760 con 0x7fbde00039b0 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52208/0 (socket says 172.21.15.123:52208) 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 -- 172.21.15.123:0/2572071963 learned_addr learned my addr 172.21.15.123:0/2572071963 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 0x7fbdfc158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdfa800640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc0d0920 0x7fbdfc163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 -- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 msgr2=0x7fbdfc158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 0x7fbdfc158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 -- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc0d0920 msgr2=0x7fbdfc163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc0d0920 0x7fbdfc163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9400640 1 -- 172.21.15.123:0/2572071963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc15ac90 con 0x7fbdfc142960 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.281+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 0x7fbdfc158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdf9400640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00cbf0 tx=0x7fbdec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec018070 con 0x7fbdfc142960 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec004d60 con 0x7fbdfc142960 2024-04-25T19:04:01.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00ad20 con 0x7fbdfc142960 2024-04-25T19:04:01.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc15af70 con 0x7fbdfc142960 2024-04-25T19:04:01.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.282+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc15b530 con 0x7fbdfc142960 2024-04-25T19:04:01.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.283+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdfc050740 con 0x7fbdfc142960 2024-04-25T19:04:01.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.285+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbdec0070d0 con 0x7fbdfc142960 2024-04-25T19:04:01.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.285+0000 7fbdf2000640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 0x7fbdd80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.286+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fbdec094f10 con 0x7fbdfc142960 2024-04-25T19:04:01.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.286+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 0x7fbdd80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.287+0000 7fbdf9e00640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 0x7fbdd80769f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde4033000 tx=0x7fbde4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.290+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdec05f320 con 0x7fbdfc142960 2024-04-25T19:04:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.490+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbdfc070f30 con 0x7fbdfc142960 2024-04-25T19:04:01.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.491+0000 7fbdf2000640 1 -- 172.21.15.123:0/2572071963 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fbdec067310 con 0x7fbdfc142960 2024-04-25T19:04:01.493 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:04:01.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 msgr2=0x7fbdd80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 0x7fbdd80769f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde4033000 tx=0x7fbde4008030 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 msgr2=0x7fbdfc0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00cbf0 tx=0x7fbdec00a430 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 shutdown_connections 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbdd80744e0 0x7fbdd80769f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdfc0d0920 0x7fbdfc163010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdfc142960 0x7fbdfc0d03e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 --2- 172.21.15.123:0/2572071963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbde00039b0 0x7fbdfc158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.493+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 >> 172.21.15.123:0/2572071963 conn(0x7fbdfc073090 msgr2=0x7fbdfc0af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.494+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 shutdown_connections 2024-04-25T19:04:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.494+0000 7fbdfb200640 1 -- 172.21.15.123:0/2572071963 wait complete. 2024-04-25T19:04:01.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:04:01.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.788+0000 7efcbd400640 1 Processor -- start 2024-04-25T19:04:01.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.788+0000 7efcbd400640 1 -- start start 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcbd400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcb806dc90 0x7efcb80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcbd400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb80d1430 con 0x7efcb80b0700 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcbd400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb8070980 con 0x7efcb815df10 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcbd400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efcb8070f40 con 0x7efcb806dc90 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcaf400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efcb80b0700 0x7efcb806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54534/0 (socket says 172.21.15.123:54534) 2024-04-25T19:04:01.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.791+0000 7efcaf400640 1 -- 172.21.15.123:0/2341954968 learned_addr learned my addr 172.21.15.123:0/2341954968 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcbca00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcb806dc90 0x7efcb80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491178274 0 0) 0x7efcb80d1430 con 0x7efcb80b0700 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc9c003520 con 0x7efcb80b0700 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4065194886 0 0) 0x7efc9c003520 con 0x7efcb80b0700 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcb806dc90 msgr2=0x7efcb80702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcb806dc90 0x7efcb80702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 >> v1:172.21.15.146:6789/0 conn(0x7efcb815df10 legacy=0x7efcb8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb8070c60 con 0x7efcb80b0700 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7efca0001670 con 0x7efcb80b0700 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 >> v1:172.21.15.123:6789/0 conn(0x7efcb80b0700 legacy=0x7efcb806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0039e0 0x7efc9c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc9c004370 0x7efc9c0048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc9c0050c0 0x7efc9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c00bf30 con 0x7efc9c0050c0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c00c0e0 con 0x7efc9c004370 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.792+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc9c00c2e0 con 0x7efc9c0039e0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc9c0050c0 0x7efc9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcb7e00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0039e0 0x7efc9c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcaf400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc9c004370 0x7efc9c0048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 -- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0039e0 msgr2=0x7efc9c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0039e0 0x7efc9c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 -- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc9c004370 msgr2=0x7efc9c0048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc9c004370 0x7efc9c0048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcbca00640 1 -- 172.21.15.123:0/2341954968 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c003520 con 0x7efc9c0050c0 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.793+0000 7efcaf400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc9c004370 0x7efc9c0048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:01.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.794+0000 7efcbca00640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc9c0050c0 0x7efc9c00b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b6820 tx=0x7efc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d207f350cb04bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.794+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0009c10 con 0x7efc9c0050c0 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.794+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb0036070 con 0x7efc9c0050c0 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.794+0000 7efcb7400640 1 -- 172.21.15.123:0/2341954968 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00317f0 con 0x7efc9c0050c0 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 -- 172.21.15.123:0/2341954968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc9c0050c0 msgr2=0x7efc9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 --2- 172.21.15.123:0/2341954968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc9c0050c0 0x7efc9c00b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b6820 tx=0x7efc9c005830 comp rx=0 tx=0).stop 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbca00640 1 -- 172.21.15.123:0/2341954968 reap_dead start 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 -- 172.21.15.123:0/2341954968 shutdown_connections 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 -- 172.21.15.123:0/2341954968 >> 172.21.15.123:0/2341954968 conn(0x7efcb8073090 msgr2=0x7efcb80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 -- 172.21.15.123:0/2341954968 shutdown_connections 2024-04-25T19:04:01.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.795+0000 7efcbd400640 1 -- 172.21.15.123:0/2341954968 wait complete. 2024-04-25T19:04:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.798+0000 7efcbd400640 1 Processor -- start 2024-04-25T19:04:01.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.798+0000 7efcbd400640 1 -- start start 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0050c0 0x7efcb8160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 0x7efcb81f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efcb806dc90 con 0x7efcb81588d0 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8161e90 con 0x7efcb814fa80 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.799+0000 7efcbd400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efcb806de10 con 0x7efc9c0050c0 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52218/0 (socket says 172.21.15.123:52218) 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 -- 172.21.15.123:0/3925211197 learned_addr learned my addr 172.21.15.123:0/3925211197 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcaf400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 0x7efcb81f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcb7e00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0050c0 0x7efcb8160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 -- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0050c0 msgr2=0x7efcb8160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0050c0 0x7efcb8160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 -- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 msgr2=0x7efcb81f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 0x7efcb81f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcbca00640 1 -- 172.21.15.123:0/3925211197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb8157830 con 0x7efcb81588d0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.800+0000 7efcaf400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 0x7efcb81f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcbca00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e230 tx=0x7efcb0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0009c10 con 0x7efcb81588d0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb0045430 con 0x7efcb81588d0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0045820 con 0x7efcb81588d0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb8157b10 con 0x7efcb81588d0 2024-04-25T19:04:01.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.801+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81580d0 con 0x7efcb81588d0 2024-04-25T19:04:01.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.802+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb8050740 con 0x7efcb81588d0 2024-04-25T19:04:01.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.803+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcb0005d10 con 0x7efcb81588d0 2024-04-25T19:04:01.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.804+0000 7efcb4c00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 0x7efc940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:01.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.804+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7efcb00b6190 con 0x7efcb81588d0 2024-04-25T19:04:01.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.805+0000 7efcb7e00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 0x7efc940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:01.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.806+0000 7efcb7e00640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 0x7efc940769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efca8007fd0 tx=0x7efca8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:01.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:01.809+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcb0080510 con 0x7efcb81588d0 2024-04-25T19:04:02.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.007+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efcb8074190 con 0x7efcb81588d0 2024-04-25T19:04:02.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.008+0000 7efcb4c00640 1 -- 172.21.15.123:0/3925211197 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7efcb007fe10 con 0x7efcb81588d0 2024-04-25T19:04:02.010 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:04:02.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 msgr2=0x7efc940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 0x7efc940769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efca8007fd0 tx=0x7efca8009040 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 msgr2=0x7efcb80b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e230 tx=0x7efcb0009750 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 shutdown_connections 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efc940744b0 0x7efc940769c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb81588d0 0x7efcb80b5b20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcb814fa80 0x7efcb81f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 --2- 172.21.15.123:0/3925211197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc9c0050c0 0x7efcb8160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.011+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 >> 172.21.15.123:0/3925211197 conn(0x7efcb8073090 msgr2=0x7efcb8155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.012+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 shutdown_connections 2024-04-25T19:04:02.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.012+0000 7efcbd400640 1 -- 172.21.15.123:0/3925211197 wait complete. 2024-04-25T19:04:02.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:04:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.316+0000 7f70a6400640 1 Processor -- start 2024-04-25T19:04:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.316+0000 7f70a6400640 1 -- start start 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f70a6400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a00b5ab0 0x7f70a01646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f70a6400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70a0161100 0x7f70a00cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f70a6400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a0163c50 con 0x7f70a00d2e70 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f70a6400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f70a0164260 con 0x7f70a0161100 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f70a6400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70a0161510 con 0x7f70a00b5ab0 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f709fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70a0161100 0x7f70a00cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f709f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f70a00d2e70 0x7f70a00b53a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54542/0 (socket says 172.21.15.123:54542) 2024-04-25T19:04:02.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.318+0000 7f709f400640 1 -- 172.21.15.123:0/532175456 learned_addr learned my addr 172.21.15.123:0/532175456 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709ea00640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a00b5ab0 0x7f70a01646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709fe00640 1 -- 172.21.15.123:0/532175456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a00b5ab0 msgr2=0x7f70a01646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709fe00640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a00b5ab0 0x7f70a01646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709fe00640 1 -- 172.21.15.123:0/532175456 >> v1:172.21.15.123:6789/0 conn(0x7f70a00d2e70 legacy=0x7f70a00b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709fe00640 1 -- 172.21.15.123:0/532175456 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a00cfa20 con 0x7f70a0161100 2024-04-25T19:04:02.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.319+0000 7f709e000640 1 -- 172.21.15.123:0/532175456 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696341119 0 0) 0x7f70a0163c50 con 0x7f70a00d2e70 2024-04-25T19:04:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.320+0000 7f709fe00640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a0161100 0x7f70a00cf390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70a01643e0 tx=0x7f709802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea85397df6f1db1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.320+0000 7f709e000640 1 -- 172.21.15.123:0/532175456 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098007740 con 0x7f70a0161100 2024-04-25T19:04:02.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.320+0000 7f709e000640 1 -- 172.21.15.123:0/532175456 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70980078e0 con 0x7f70a0161100 2024-04-25T19:04:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.320+0000 7f709e000640 1 -- 172.21.15.123:0/532175456 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098007cb0 con 0x7f70a0161100 2024-04-25T19:04:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 -- 172.21.15.123:0/532175456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a0161100 msgr2=0x7f70a00cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a0161100 0x7f70a00cf390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70a01643e0 tx=0x7f709802a5b0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 -- 172.21.15.123:0/532175456 shutdown_connections 2024-04-25T19:04:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a0161100 0x7f70a00cf390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 --2- 172.21.15.123:0/532175456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a00b5ab0 0x7f70a01646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 -- 172.21.15.123:0/532175456 >> 172.21.15.123:0/532175456 conn(0x7f70a00d03a0 msgr2=0x7f70a00d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 -- 172.21.15.123:0/532175456 shutdown_connections 2024-04-25T19:04:02.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.321+0000 7f70a6400640 1 -- 172.21.15.123:0/532175456 wait complete. 2024-04-25T19:04:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.324+0000 7f70a6400640 1 Processor -- start 2024-04-25T19:04:02.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.324+0000 7f70a6400640 1 -- start start 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a0161100 0x7f70a0070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 0x7f70a014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70a0163ef0 con 0x7f70a00d2e70 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f70a00cffe0 con 0x7f70a00aeaf0 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f70a6400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70a015f7a0 con 0x7f70a0161100 2024-04-25T19:04:02.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.325+0000 7f709f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52236/0 (socket says 172.21.15.123:52236) 2024-04-25T19:04:02.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 -- 172.21.15.123:0/2399044443 learned_addr learned my addr 172.21.15.123:0/2399044443 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709ea00640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a0161100 0x7f70a0070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709fe00640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 0x7f70a014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 -- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a0161100 msgr2=0x7f70a0070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a0161100 0x7f70a0070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 -- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 msgr2=0x7f70a014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 0x7f70a014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 -- 172.21.15.123:0/2399044443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a014d4a0 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709fe00640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 0x7f70a014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.326+0000 7f709f400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f709000c410 tx=0x7f709000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.327+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900074e0 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.327+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70900047e0 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.327+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a006fd20 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.327+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090004bd0 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.327+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a00702e0 con 0x7f70a00d2e70 2024-04-25T19:04:02.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.328+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a0076a40 con 0x7f70a00d2e70 2024-04-25T19:04:02.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.329+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7090011070 con 0x7f70a00d2e70 2024-04-25T19:04:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.330+0000 7f7097400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 0x7f70800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.330+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f70900950c0 con 0x7f70a00d2e70 2024-04-25T19:04:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.330+0000 7f709ea00640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 0x7f70800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.331+0000 7f709ea00640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 0x7f70800769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.335+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70a014d4a0 con 0x7f70a00d2e70 2024-04-25T19:04:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.519+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70a00d2c50 con 0x7f70a00d2e70 2024-04-25T19:04:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.520+0000 7f7097400640 1 -- 172.21.15.123:0/2399044443 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f709005f570 con 0x7f70a00d2e70 2024-04-25T19:04:02.522 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:04:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 msgr2=0x7f70800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 0x7f70800769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c008040 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 msgr2=0x7f70a006f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f709000c410 tx=0x7f709000c8d0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 shutdown_connections 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f70800744e0 0x7f70800769f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70a00aeaf0 0x7f70a014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70a0161100 0x7f70a0070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 --2- 172.21.15.123:0/2399044443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a00d2e70 0x7f70a006f230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 >> 172.21.15.123:0/2399044443 conn(0x7f70a00d03a0 msgr2=0x7f70a00d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 shutdown_connections 2024-04-25T19:04:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.523+0000 7f70a6400640 1 -- 172.21.15.123:0/2399044443 wait complete. 2024-04-25T19:04:02.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:04:02.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.817+0000 7fcd8a600640 1 Processor -- start 2024-04-25T19:04:02.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.818+0000 7fcd8a600640 1 -- start start 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.820+0000 7fcd8a600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd840cd140 0x7fcd8415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.820+0000 7fcd8a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.820+0000 7fcd8a600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd840cc5f0 con 0x7fcd8406daa0 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.820+0000 7fcd8a600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8406deb0 con 0x7fcd840cd140 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.820+0000 7fcd8a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8406d610 con 0x7fcd84162540 2024-04-25T19:04:02.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd7aa00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd840cd140 0x7fcd8415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:47030/0 (socket says 172.21.15.123:47030) 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 -- 172.21.15.123:0/1502555811 learned_addr learned my addr 172.21.15.123:0/1502555811 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 -- 172.21.15.123:0/1502555811 >> v2:172.21.15.146:3300/0 conn(0x7fcd840cd140 msgr2=0x7fcd8415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 --2- 172.21.15.123:0/1502555811 >> v2:172.21.15.146:3300/0 conn(0x7fcd840cd140 0x7fcd8415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 -- 172.21.15.123:0/1502555811 >> v1:172.21.15.146:6789/0 conn(0x7fcd8406daa0 legacy=0x7fcd840cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.821+0000 7fcd83e00640 1 -- 172.21.15.123:0/1502555811 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8406ca00 con 0x7fcd84162540 2024-04-25T19:04:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.823+0000 7fcd83e00640 1 --2- 172.21.15.123:0/1502555811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009e50 tx=0x7fcd7c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38c391a07c38477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.823+0000 7fcd82a00640 1 -- 172.21.15.123:0/1502555811 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c036070 con 0x7fcd84162540 2024-04-25T19:04:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.823+0000 7fcd82a00640 1 -- 172.21.15.123:0/1502555811 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c031600 con 0x7fcd84162540 2024-04-25T19:04:02.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.823+0000 7fcd82a00640 1 -- 172.21.15.123:0/1502555811 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0319d0 con 0x7fcd84162540 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 -- 172.21.15.123:0/1502555811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 msgr2=0x7fcd840af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1502555811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009e50 tx=0x7fcd7c02a710 comp rx=0 tx=0).stop 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 -- 172.21.15.123:0/1502555811 shutdown_connections 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1502555811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84162540 0x7fcd840af040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1502555811 >> v2:172.21.15.146:3300/0 conn(0x7fcd840cd140 0x7fcd8415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 -- 172.21.15.123:0/1502555811 >> 172.21.15.123:0/1502555811 conn(0x7fcd840ce390 msgr2=0x7fcd840ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 -- 172.21.15.123:0/1502555811 shutdown_connections 2024-04-25T19:04:02.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.824+0000 7fcd8a600640 1 -- 172.21.15.123:0/1502555811 wait complete. 2024-04-25T19:04:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.827+0000 7fcd8a600640 1 Processor -- start 2024-04-25T19:04:02.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.827+0000 7fcd8a600640 1 -- start start 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 0x7fcd84153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84156c50 0x7fcd840b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8406d2a0 con 0x7fcd84162540 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8406cfc0 con 0x7fcd8406daa0 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.828+0000 7fcd8a600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcd84162950 con 0x7fcd84156c50 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52250/0 (socket says 172.21.15.123:52250) 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 -- 172.21.15.123:0/1903157037 learned_addr learned my addr 172.21.15.123:0/1903157037 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:02.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd83400640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 0x7fcd84153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd83e00640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84156c50 0x7fcd840b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 -- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84156c50 msgr2=0x7fcd840b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84156c50 0x7fcd840b6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 -- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 msgr2=0x7fcd84153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 0x7fcd84153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd7aa00640 1 -- 172.21.15.123:0/1903157037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd840b4870 con 0x7fcd84162540 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.829+0000 7fcd83400640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 0x7fcd84153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:02.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd7aa00640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e8a0 tx=0x7fcd6c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c017070 con 0x7fcd84162540 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00ee90 con 0x7fcd84162540 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd841573c0 con 0x7fcd84162540 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00cb00 con 0x7fcd84162540 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.830+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8415b870 con 0x7fcd84162540 2024-04-25T19:04:02.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.831+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd84157130 con 0x7fcd84162540 2024-04-25T19:04:02.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.832+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd6c01a070 con 0x7fcd84162540 2024-04-25T19:04:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.834+0000 7fcd7be00640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 0x7fcd600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.834+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fcd6c095430 con 0x7fcd84162540 2024-04-25T19:04:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.834+0000 7fcd83400640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 0x7fcd600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:02.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.835+0000 7fcd83400640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 0x7fcd600769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd8409fb30 tx=0x7fcd74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:02.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:02.839+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd6c05f7b0 con 0x7fcd84162540 2024-04-25T19:04:03.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.040+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcd84076a40 con 0x7fcd84162540 2024-04-25T19:04:03.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.041+0000 7fcd7be00640 1 -- 172.21.15.123:0/1903157037 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fcd6c05f0b0 con 0x7fcd84162540 2024-04-25T19:04:03.043 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 msgr2=0x7fcd600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 0x7fcd600769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd8409fb30 tx=0x7fcd74009040 comp rx=0 tx=0).stop 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 msgr2=0x7fcd840b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e8a0 tx=0x7fcd6c00ed60 comp rx=0 tx=0).stop 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 shutdown_connections 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fcd600744e0 0x7fcd600769f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd84156c50 0x7fcd840b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd84162540 0x7fcd840b6680 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 --2- 172.21.15.123:0/1903157037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd8406daa0 0x7fcd84153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 >> 172.21.15.123:0/1903157037 conn(0x7fcd840ce390 msgr2=0x7fcd841eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 shutdown_connections 2024-04-25T19:04:03.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.044+0000 7fcd8a600640 1 -- 172.21.15.123:0/1903157037 wait complete. 2024-04-25T19:04:03.073 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:04:03.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.334+0000 7f4df2a00640 1 Processor -- start 2024-04-25T19:04:03.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.334+0000 7f4df2a00640 1 -- start start 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec13fbf0 0x7f4dec06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dec06dac0 0x7f4dec142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec06ded0 con 0x7f4dec142a70 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4dec0b5650 con 0x7f4dec06dac0 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4dec0b5c70 con 0x7f4dec13fbf0 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dec06dac0 0x7f4dec142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dec06dac0 0x7f4dec142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52266/0 (socket says 172.21.15.123:52266) 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df2000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4dec142a70 0x7f4dec1607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54558/0 (socket says 172.21.15.123:54558) 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 -- 172.21.15.123:0/3696672742 learned_addr learned my addr 172.21.15.123:0/3696672742 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df1600640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec13fbf0 0x7f4dec06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 -- 172.21.15.123:0/3696672742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec13fbf0 msgr2=0x7f4dec06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec13fbf0 0x7f4dec06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 -- 172.21.15.123:0/3696672742 >> v1:172.21.15.123:6789/0 conn(0x7f4dec142a70 legacy=0x7f4dec1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.336+0000 7f4df0c00640 1 -- 172.21.15.123:0/3696672742 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec0b5f50 con 0x7f4dec06dac0 2024-04-25T19:04:03.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.337+0000 7f4df0c00640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec06dac0 0x7f4dec142530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d150 tx=0x7f4dd802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d1e936030671db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.338+0000 7f4debe00640 1 -- 172.21.15.123:0/3696672742 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8009600 con 0x7f4dec06dac0 2024-04-25T19:04:03.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.338+0000 7f4debe00640 1 -- 172.21.15.123:0/3696672742 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd8037070 con 0x7f4dec06dac0 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.338+0000 7f4debe00640 1 -- 172.21.15.123:0/3696672742 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80099c0 con 0x7f4dec06dac0 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.338+0000 7f4df2a00640 1 -- 172.21.15.123:0/3696672742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec06dac0 msgr2=0x7f4dec142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.338+0000 7f4df2a00640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec06dac0 0x7f4dec142530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d150 tx=0x7f4dd802c920 comp rx=0 tx=0).stop 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 -- 172.21.15.123:0/3696672742 shutdown_connections 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec06dac0 0x7f4dec142530 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 --2- 172.21.15.123:0/3696672742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec13fbf0 0x7f4dec06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 -- 172.21.15.123:0/3696672742 >> 172.21.15.123:0/3696672742 conn(0x7f4dec073200 msgr2=0x7f4dec073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 -- 172.21.15.123:0/3696672742 shutdown_connections 2024-04-25T19:04:03.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.339+0000 7f4df2a00640 1 -- 172.21.15.123:0/3696672742 wait complete. 2024-04-25T19:04:03.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.341+0000 7f4df2a00640 1 Processor -- start 2024-04-25T19:04:03.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.341+0000 7f4df2a00640 1 -- start start 2024-04-25T19:04:03.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec06dac0 0x7f4dec140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dec13fbf0 0x7f4dec1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0b6a30 con 0x7f4dec142a70 2024-04-25T19:04:03.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0b6510 con 0x7f4dec13fbf0 2024-04-25T19:04:03.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.342+0000 7f4df2a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4dec142e80 con 0x7f4dec06dac0 2024-04-25T19:04:03.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52270/0 (socket says 172.21.15.123:52270) 2024-04-25T19:04:03.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 -- 172.21.15.123:0/384016869 learned_addr learned my addr 172.21.15.123:0/384016869 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 -- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec06dac0 msgr2=0x7f4dec140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df1600640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec06dac0 0x7f4dec140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec06dac0 0x7f4dec140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df0c00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dec13fbf0 0x7f4dec1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 -- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dec13fbf0 msgr2=0x7f4dec1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dec13fbf0 0x7f4dec1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.343+0000 7f4df2000640 1 -- 172.21.15.123:0/384016869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec141940 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4df2000640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c3f0 tx=0x7f4de400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40074e0 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de4007680 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40049e0 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec1408a0 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.344+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec140e60 con 0x7f4dec142a70 2024-04-25T19:04:03.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.345+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dec050740 con 0x7f4dec142a70 2024-04-25T19:04:03.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.346+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4de401a020 con 0x7f4dec142a70 2024-04-25T19:04:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.347+0000 7f4de9600640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 0x7f4dcc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.347+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f4de4094cf0 con 0x7f4dec142a70 2024-04-25T19:04:03.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.347+0000 7f4df1600640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 0x7f4dcc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.348+0000 7f4df1600640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 0x7f4dcc0769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b7c0 tx=0x7f4ddc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.352+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dec06ded0 con 0x7f4dec142a70 2024-04-25T19:04:03.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.564+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4dec0cfb70 con 0x7f4dec142a70 2024-04-25T19:04:03.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.564+0000 7f4de9600640 1 -- 172.21.15.123:0/384016869 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f4de405f100 con 0x7f4dec142a70 2024-04-25T19:04:03.566 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:04:03.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 msgr2=0x7f4dcc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 0x7f4dcc0769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b7c0 tx=0x7f4ddc008040 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 msgr2=0x7f4dec155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c3f0 tx=0x7f4de400c8b0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 shutdown_connections 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4dcc0744e0 0x7f4dcc0769f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dec142a70 0x7f4dec155e30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dec13fbf0 0x7f4dec1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 --2- 172.21.15.123:0/384016869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4dec06dac0 0x7f4dec140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.567+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 >> 172.21.15.123:0/384016869 conn(0x7f4dec073200 msgr2=0x7f4dec0b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.568+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 shutdown_connections 2024-04-25T19:04:03.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:03.568+0000 7f4df2a00640 1 -- 172.21.15.123:0/384016869 wait complete. 2024-04-25T19:04:03.594 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T19:04:03.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:03.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T19:04:03.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:04:03.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:03.689 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T19:04:03.737 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T19:04:03.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:04.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.009+0000 7f3f82c00640 1 Processor -- start 2024-04-25T19:04:04.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.009+0000 7f3f82c00640 1 -- start start 2024-04-25T19:04:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.011+0000 7f3f82c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f7c0d22e0 0x7f3f7c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.011+0000 7f3f82c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 0x7f3f7c164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.011+0000 7f3f82c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7c0b3c30 con 0x7f3f7c0cfd10 2024-04-25T19:04:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.011+0000 7f3f82c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7c164fc0 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.011+0000 7f3f82c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7c0b39e0 con 0x7f3f7c06f500 2024-04-25T19:04:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3f7c0cfd10 0x7f3f7c0d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54562/0 (socket says 172.21.15.123:54562) 2024-04-25T19:04:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7be00640 1 -- 172.21.15.123:0/3370819935 learned_addr learned my addr 172.21.15.123:0/3370819935 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3370819935 >> v2:172.21.15.123:3300/0 conn(0x7f3f7c0d22e0 0x7f3f7c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7b400640 1 -- 172.21.15.123:0/3370819935 >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 msgr2=0x7f3f7c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f80c00640 1 --2- 172.21.15.123:0/3370819935 >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 0x7f3f7c164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3370819935 >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 0x7f3f7c164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7aa00640 1 -- 172.21.15.123:0/3370819935 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59367070 0 0) 0x7f3f7c0b3c30 con 0x7f3f7c0cfd10 2024-04-25T19:04:04.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7b400640 1 -- 172.21.15.123:0/3370819935 >> v1:172.21.15.123:6789/0 conn(0x7f3f7c0cfd10 legacy=0x7f3f7c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f7b400640 1 -- 172.21.15.123:0/3370819935 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c0b3f10 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.012+0000 7f3f80c00640 1 --2- 172.21.15.123:0/3370819935 >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 0x7f3f7c164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.013+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3370819935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c0d22e0 0x7f3f7c0b4cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0d3970 tx=0x7f3f6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb171a24e7125163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.013+0000 7f3f7aa00640 1 -- 172.21.15.123:0/3370819935 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037070 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f7aa00640 1 -- 172.21.15.123:0/3370819935 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68032440 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f7aa00640 1 -- 172.21.15.123:0/3370819935 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6802ae60 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 -- 172.21.15.123:0/3370819935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c0d22e0 msgr2=0x7f3f7c0b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3370819935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c0d22e0 0x7f3f7c0b4cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0d3970 tx=0x7f3f6802a110 comp rx=0 tx=0).stop 2024-04-25T19:04:04.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 -- 172.21.15.123:0/3370819935 shutdown_connections 2024-04-25T19:04:04.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3370819935 >> v2:172.21.15.146:3300/0 conn(0x7f3f7c06f500 0x7f3f7c164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3370819935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c0d22e0 0x7f3f7c0b4cd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 -- 172.21.15.123:0/3370819935 >> 172.21.15.123:0/3370819935 conn(0x7f3f7c09b4a0 msgr2=0x7f3f7c09b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:04.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 -- 172.21.15.123:0/3370819935 shutdown_connections 2024-04-25T19:04:04.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.014+0000 7f3f82c00640 1 -- 172.21.15.123:0/3370819935 wait complete. 2024-04-25T19:04:04.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.017+0000 7f3f82c00640 1 Processor -- start 2024-04-25T19:04:04.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.017+0000 7f3f82c00640 1 -- start start 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 0x7f3f7c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f7c0d22e0 0x7f3f7c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0b47b0 con 0x7f3f7c06f500 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0b44d0 con 0x7f3f7c0cfd10 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f82c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c06f910 con 0x7f3f7c0d22e0 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.018+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52286/0 (socket says 172.21.15.123:52286) 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 -- 172.21.15.123:0/3153883499 learned_addr learned my addr 172.21.15.123:0/3153883499 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 0x7f3f7c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 -- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f7c0d22e0 msgr2=0x7f3f7c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f7c0d22e0 0x7f3f7c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 -- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 msgr2=0x7f3f7c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 0x7f3f7c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 -- 172.21.15.123:0/3153883499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c0c72b0 con 0x7f3f7c06f500 2024-04-25T19:04:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 0x7f3f7c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f7be00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c780 tx=0x7f3f6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.019+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c018070 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.020+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c00ce30 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.020+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c0af780 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.020+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00a800 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.020+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c0c6d40 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.021+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c076a40 con 0x7f3f7c06f500 2024-04-25T19:04:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.022+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3f6c01b070 con 0x7f3f7c06f500 2024-04-25T19:04:04.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.023+0000 7f3f73e00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 0x7f3f5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.023+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f3f6c094bc0 con 0x7f3f7c06f500 2024-04-25T19:04:04.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.023+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 0x7f3f5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.024+0000 7f3f7b400640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 0x7f3f5c0769f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034000 tx=0x7f3f6802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.028+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f6c05ef40 con 0x7f3f7c06f500 2024-04-25T19:04:04.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.277+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f7c0cfac0 con 0x7f3f7c06f500 2024-04-25T19:04:04.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.278+0000 7f3f73e00640 1 -- 172.21.15.123:0/3153883499 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f3f6c05e840 con 0x7f3f7c06f500 2024-04-25T19:04:04.280 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:04.280 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:04:04.280 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 msgr2=0x7f3f5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 0x7f3f5c0769f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034000 tx=0x7f3f6802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 msgr2=0x7f3f7c15cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c780 tx=0x7f3f6c00cc40 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 shutdown_connections 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f3f5c0744e0 0x7f3f5c0769f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f7c0d22e0 0x7f3f7c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f7c0cfd10 0x7f3f7c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 --2- 172.21.15.123:0/3153883499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f7c06f500 0x7f3f7c15cd10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 >> 172.21.15.123:0/3153883499 conn(0x7f3f7c09b4a0 msgr2=0x7f3f7c164000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:04.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 shutdown_connections 2024-04-25T19:04:04.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.281+0000 7f3f82c00640 1 -- 172.21.15.123:0/3153883499 wait complete. 2024-04-25T19:04:04.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.560+0000 7fd3a5200640 1 Processor -- start 2024-04-25T19:04:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.561+0000 7fd3a5200640 1 -- start start 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd3a5200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0147880 0x7fd3a0147c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd3a5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3a0164c40 0x7fd3a006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd3a5200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a0165360 con 0x7fd3a0164830 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd3a5200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a0165640 con 0x7fd3a0164c40 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd3a5200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00d02a0 con 0x7fd3a0147880 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd39ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd3a0164830 0x7fd3a00d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54574/0 (socket says 172.21.15.123:54574) 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd39ea00640 1 -- 172.21.15.123:0/1414244 learned_addr learned my addr 172.21.15.123:0/1414244 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.563+0000 7fd39fe00640 1 --2- 172.21.15.123:0/1414244 >> v2:172.21.15.123:3300/0 conn(0x7fd3a0164c40 0x7fd3a006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39f400640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0147880 0x7fd3a0147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39e000640 1 -- 172.21.15.123:0/1414244 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4046043759 0 0) 0x7fd3a0165360 con 0x7fd3a0164830 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39e000640 1 -- 172.21.15.123:0/1414244 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd380003520 con 0x7fd3a0164830 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39fe00640 1 -- 172.21.15.123:0/1414244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0147880 msgr2=0x7fd3a0147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39fe00640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0147880 0x7fd3a0147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39fe00640 1 -- 172.21.15.123:0/1414244 >> v1:172.21.15.123:6789/0 conn(0x7fd3a0164830 legacy=0x7fd3a00d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.564+0000 7fd39fe00640 1 -- 172.21.15.123:0/1414244 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00d0580 con 0x7fd3a0164c40 2024-04-25T19:04:04.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.565+0000 7fd39fe00640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0164c40 0x7fd3a006d580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097a0 tx=0x7fd39802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da63d7a1b837ab55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.565+0000 7fd39e000640 1 -- 172.21.15.123:0/1414244 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398037070 con 0x7fd3a0164c40 2024-04-25T19:04:04.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.565+0000 7fd39e000640 1 -- 172.21.15.123:0/1414244 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39802ae40 con 0x7fd3a0164c40 2024-04-25T19:04:04.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.565+0000 7fd39e000640 1 -- 172.21.15.123:0/1414244 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398032640 con 0x7fd3a0164c40 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 -- 172.21.15.123:0/1414244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0164c40 msgr2=0x7fd3a006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0164c40 0x7fd3a006d580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097a0 tx=0x7fd39802a0f0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 -- 172.21.15.123:0/1414244 shutdown_connections 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0164c40 0x7fd3a006d580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 --2- 172.21.15.123:0/1414244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0147880 0x7fd3a0147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 -- 172.21.15.123:0/1414244 >> 172.21.15.123:0/1414244 conn(0x7fd3a0073200 msgr2=0x7fd3a0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 -- 172.21.15.123:0/1414244 shutdown_connections 2024-04-25T19:04:04.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.566+0000 7fd3a5200640 1 -- 172.21.15.123:0/1414244 wait complete. 2024-04-25T19:04:04.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.569+0000 7fd3a5200640 1 Processor -- start 2024-04-25T19:04:04.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.569+0000 7fd3a5200640 1 -- start start 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a0164830 0x7fd3a0146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0164c40 0x7fd3a015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00ce390 con 0x7fd3a0147880 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0165050 con 0x7fd3a0164830 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.571+0000 7fd3a5200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00ce510 con 0x7fd3a0164c40 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52294/0 (socket says 172.21.15.123:52294) 2024-04-25T19:04:04.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 -- 172.21.15.123:0/3826969948 learned_addr learned my addr 172.21.15.123:0/3826969948 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 -- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0164c40 msgr2=0x7fd3a015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0164c40 0x7fd3a015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 -- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a0164830 msgr2=0x7fd3a0146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a0164830 0x7fd3a0146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 -- 172.21.15.123:0/3826969948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a01ecc20 con 0x7fd3a0147880 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd39f400640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd39000d5d0 tx=0x7fd39000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.572+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390004240 con 0x7fd3a0147880 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.573+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3900043e0 con 0x7fd3a0147880 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.573+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01ecf00 con 0x7fd3a0147880 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.573+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390003830 con 0x7fd3a0147880 2024-04-25T19:04:04.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.573+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01ed4c0 con 0x7fd3a0147880 2024-04-25T19:04:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.574+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a00d13b0 con 0x7fd3a0147880 2024-04-25T19:04:04.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.575+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3900039d0 con 0x7fd3a0147880 2024-04-25T19:04:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.576+0000 7fd397400640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 0x7fd37c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.576+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fd3900958c0 con 0x7fd3a0147880 2024-04-25T19:04:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.576+0000 7fd39ea00640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 0x7fd37c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.577+0000 7fd39ea00640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 0x7fd37c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3880012e0 tx=0x7fd3880015b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:04.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.582+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3a00d13b0 con 0x7fd3a0147880 2024-04-25T19:04:04.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.783+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3a0050740 con 0x7fd3a0147880 2024-04-25T19:04:04.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.786+0000 7fd397400640 1 -- 172.21.15.123:0/3826969948 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7fd39005fcd0 con 0x7fd3a0147880 2024-04-25T19:04:04.788 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:04.788 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:04.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 msgr2=0x7fd37c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 0x7fd37c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3880012e0 tx=0x7fd3880015b0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 msgr2=0x7fd3a0161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd39000d5d0 tx=0x7fd39000da90 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 shutdown_connections 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd37c0744b0 0x7fd37c0769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a0164c40 0x7fd3a015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a0164830 0x7fd3a0146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 --2- 172.21.15.123:0/3826969948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3a0147880 0x7fd3a0161a00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.788+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 >> 172.21.15.123:0/3826969948 conn(0x7fd3a0073200 msgr2=0x7fd3a0166270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.789+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 shutdown_connections 2024-04-25T19:04:04.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:04.789+0000 7fd3a5200640 1 -- 172.21.15.123:0/3826969948 wait complete. 2024-04-25T19:04:04.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:02:37.807902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:02:38.536776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T19:02:42.111213+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T19:04:04.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:04:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.087+0000 7fa859800640 1 Processor -- start 2024-04-25T19:04:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.087+0000 7fa859800640 1 -- start start 2024-04-25T19:04:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.089+0000 7fa859800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8540cfff0 0x7fa85415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.089+0000 7fa859800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85415e2e0 con 0x7fa85406dc90 2024-04-25T19:04:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.089+0000 7fa859800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540d05f0 con 0x7fa854143040 2024-04-25T19:04:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.089+0000 7fa859800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540d1200 con 0x7fa8540cfff0 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa858e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa85406dc90 0x7fa8540cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:54580/0 (socket says 172.21.15.123:54580) 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa853e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8540cfff0 0x7fa85415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa858e00640 1 -- 172.21.15.123:0/3762252131 learned_addr learned my addr 172.21.15.123:0/3762252131 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa853e00640 1 -- 172.21.15.123:0/3762252131 >> v1:172.21.15.146:6789/0 conn(0x7fa854143040 legacy=0x7fa85406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa853e00640 1 -- 172.21.15.123:0/3762252131 >> v1:172.21.15.123:6789/0 conn(0x7fa85406dc90 legacy=0x7fa8540cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa853e00640 1 -- 172.21.15.123:0/3762252131 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540d08d0 con 0x7fa8540cfff0 2024-04-25T19:04:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.090+0000 7fa853400640 1 -- 172.21.15.123:0/3762252131 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773418840 0 0) 0x7fa84c001490 con 0x7fa85406dc90 2024-04-25T19:04:05.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.091+0000 7fa853e00640 1 --2- 172.21.15.123:0/3762252131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540cfff0 0x7fa85415dda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0097a0 tx=0x7fa83c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84de09660735d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.091+0000 7fa853400640 1 -- 172.21.15.123:0/3762252131 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c037070 con 0x7fa8540cfff0 2024-04-25T19:04:05.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.091+0000 7fa853400640 1 -- 172.21.15.123:0/3762252131 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c032440 con 0x7fa8540cfff0 2024-04-25T19:04:05.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.091+0000 7fa853400640 1 -- 172.21.15.123:0/3762252131 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c032810 con 0x7fa8540cfff0 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 -- 172.21.15.123:0/3762252131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540cfff0 msgr2=0x7fa85415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 --2- 172.21.15.123:0/3762252131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540cfff0 0x7fa85415dda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0097a0 tx=0x7fa83c02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 -- 172.21.15.123:0/3762252131 shutdown_connections 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 --2- 172.21.15.123:0/3762252131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540cfff0 0x7fa85415dda0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 -- 172.21.15.123:0/3762252131 >> 172.21.15.123:0/3762252131 conn(0x7fa854073090 msgr2=0x7fa8540734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 -- 172.21.15.123:0/3762252131 shutdown_connections 2024-04-25T19:04:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.092+0000 7fa859800640 1 -- 172.21.15.123:0/3762252131 wait complete. 2024-04-25T19:04:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.095+0000 7fa859800640 1 Processor -- start 2024-04-25T19:04:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.095+0000 7fa859800640 1 -- start start 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8540cfff0 0x7fa8540b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa854143040 0x7fa8540cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa854143450 con 0x7fa8540ae170 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540d0e90 con 0x7fa854143040 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.096+0000 7fa859800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa8540acfe0 con 0x7fa8540cfff0 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52316/0 (socket says 172.21.15.123:52316) 2024-04-25T19:04:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 -- 172.21.15.123:0/620892340 learned_addr learned my addr 172.21.15.123:0/620892340 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa853e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8540cfff0 0x7fa8540b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa84be00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa854143040 0x7fa8540cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 -- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8540cfff0 msgr2=0x7fa8540b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8540cfff0 0x7fa8540b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 -- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa854143040 msgr2=0x7fa8540cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa854143040 0x7fa8540cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.097+0000 7fa858e00640 1 -- 172.21.15.123:0/620892340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540aeac0 con 0x7fa8540ae170 2024-04-25T19:04:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa858e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c730 tx=0x7fa84c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c018070 con 0x7fa8540ae170 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c00ce40 con 0x7fa8540ae170 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00a860 con 0x7fa8540ae170 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8540aa390 con 0x7fa8540ae170 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.098+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8540aa950 con 0x7fa8540ae170 2024-04-25T19:04:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.099+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8540aa100 con 0x7fa8540ae170 2024-04-25T19:04:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.100+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa84c00aa00 con 0x7fa8540ae170 2024-04-25T19:04:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.101+0000 7fa850c00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 0x7fa8300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.101+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fa84c094a60 con 0x7fa8540ae170 2024-04-25T19:04:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.102+0000 7fa853e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 0x7fa8300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.102+0000 7fa853e00640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 0x7fa8300769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa83c034000 tx=0x7fa83c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.106+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8540aeac0 con 0x7fa8540ae170 2024-04-25T19:04:05.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.290+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa854050740 con 0x7fa8540ae170 2024-04-25T19:04:05.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.291+0000 7fa850c00640 1 -- 172.21.15.123:0/620892340 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fa85415e2e0 con 0x7fa8540ae170 2024-04-25T19:04:05.293 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.293+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 msgr2=0x7fa8300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.293+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 0x7fa8300769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa83c034000 tx=0x7fa83c02e000 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 msgr2=0x7fa8540ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c730 tx=0x7fa84c00cbf0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 shutdown_connections 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa8300744b0 0x7fa8300769c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8540ae170 0x7fa8540ae580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa854143040 0x7fa8540cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 --2- 172.21.15.123:0/620892340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8540cfff0 0x7fa8540b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 >> 172.21.15.123:0/620892340 conn(0x7fa854073090 msgr2=0x7fa85406cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 shutdown_connections 2024-04-25T19:04:05.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.294+0000 7fa859800640 1 -- 172.21.15.123:0/620892340 wait complete. 2024-04-25T19:04:05.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:04:05.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.576+0000 7f979ac00640 1 Processor -- start 2024-04-25T19:04:05.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.576+0000 7f979ac00640 1 -- start start 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979ac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f979415df10 0x7f97940702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979ac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97940d1430 con 0x7f979406d580 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979ac00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9794070980 con 0x7f97940b0700 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979ac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9794070f40 con 0x7f979415df10 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f9799800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f979415df10 0x7f97940702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979a200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f979406d580 0x7f979406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38860/0 (socket says 172.21.15.123:38860) 2024-04-25T19:04:05.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.579+0000 7f979a200640 1 -- 172.21.15.123:0/809720897 learned_addr learned my addr 172.21.15.123:0/809720897 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:05.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.580+0000 7f9799800640 1 -- 172.21.15.123:0/809720897 >> v1:172.21.15.146:6789/0 conn(0x7f97940b0700 legacy=0x7f9794162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:05.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.580+0000 7f9799800640 1 -- 172.21.15.123:0/809720897 >> v1:172.21.15.123:6789/0 conn(0x7f979406d580 legacy=0x7f979406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.580+0000 7f9799800640 1 -- 172.21.15.123:0/809720897 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9794070c60 con 0x7f979415df10 2024-04-25T19:04:05.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.581+0000 7f9799800640 1 --2- 172.21.15.123:0/809720897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f97940702f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f97840097a0 tx=0x7f978402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11636920e5c45a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.581+0000 7f9793e00640 1 -- 172.21.15.123:0/809720897 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784037070 con 0x7f979415df10 2024-04-25T19:04:05.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.581+0000 7f9793e00640 1 -- 172.21.15.123:0/809720897 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9784032440 con 0x7f979415df10 2024-04-25T19:04:05.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.581+0000 7f9793e00640 1 -- 172.21.15.123:0/809720897 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784032810 con 0x7f979415df10 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 -- 172.21.15.123:0/809720897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 msgr2=0x7f97940702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 --2- 172.21.15.123:0/809720897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f97940702f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f97840097a0 tx=0x7f978402a1e0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 -- 172.21.15.123:0/809720897 shutdown_connections 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 --2- 172.21.15.123:0/809720897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f97940702f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 -- 172.21.15.123:0/809720897 >> 172.21.15.123:0/809720897 conn(0x7f9794073090 msgr2=0x7f97940734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 -- 172.21.15.123:0/809720897 shutdown_connections 2024-04-25T19:04:05.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.582+0000 7f979ac00640 1 -- 172.21.15.123:0/809720897 wait complete. 2024-04-25T19:04:05.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.585+0000 7f979ac00640 1 Processor -- start 2024-04-25T19:04:05.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.585+0000 7f979ac00640 1 -- start start 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f979406d580 0x7f97940ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 0x7f979414e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97940cdfe0 con 0x7f979415df10 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9794161e90 con 0x7f979406d580 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979ac00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97940ce160 con 0x7f97940b0700 2024-04-25T19:04:05.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42794/0 (socket says 172.21.15.123:42794) 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.587+0000 7f979a200640 1 -- 172.21.15.123:0/952665703 learned_addr learned my addr 172.21.15.123:0/952665703 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9799800640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f979406d580 0x7f97940ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9798e00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 0x7f979414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 -- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 msgr2=0x7f979414e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 0x7f979414e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 -- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f979406d580 msgr2=0x7f97940ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f979406d580 0x7f97940ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 -- 172.21.15.123:0/952665703 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97940aebf0 con 0x7f979415df10 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9798e00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 0x7f979414e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979a200640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f978c00cad0 tx=0x7f978c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c017070 con 0x7f979415df10 2024-04-25T19:04:05.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c00ab70 con 0x7f979415df10 2024-04-25T19:04:05.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c012400 con 0x7f979415df10 2024-04-25T19:04:05.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.588+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97940aeed0 con 0x7f979415df10 2024-04-25T19:04:05.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.589+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97940af490 con 0x7f979415df10 2024-04-25T19:04:05.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.590+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9794050740 con 0x7f979415df10 2024-04-25T19:04:05.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.591+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f978c01a070 con 0x7f979415df10 2024-04-25T19:04:05.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.592+0000 7f9791600640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 0x7f97740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:05.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.592+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f978c0951d0 con 0x7f979415df10 2024-04-25T19:04:05.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.593+0000 7f9799800640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 0x7f97740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:05.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.593+0000 7f9799800640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 0x7f97740769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9784034000 tx=0x7f978402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:05.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.598+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f978c05ef00 con 0x7f979415df10 2024-04-25T19:04:05.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.796+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97940afc90 con 0x7f979415df10 2024-04-25T19:04:05.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.797+0000 7f9791600640 1 -- 172.21.15.123:0/952665703 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f978c063200 con 0x7f979415df10 2024-04-25T19:04:05.799 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:04:05.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.799+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 msgr2=0x7f97740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.799+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 0x7f97740769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9784034000 tx=0x7f978402e000 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.799+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 msgr2=0x7f979415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.799+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f978c00cad0 tx=0x7f978c00cf90 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 shutdown_connections 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f97740744b0 0x7f97740769c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f979415df10 0x7f979415ac00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97940b0700 0x7f979414e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 --2- 172.21.15.123:0/952665703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f979406d580 0x7f97940ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 >> 172.21.15.123:0/952665703 conn(0x7f9794073090 msgr2=0x7f979415e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 shutdown_connections 2024-04-25T19:04:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:05.800+0000 7f979ac00640 1 -- 172.21.15.123:0/952665703 wait complete. 2024-04-25T19:04:05.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:04:06.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.088+0000 7f616d400640 1 Processor -- start 2024-04-25T19:04:06.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.088+0000 7f616d400640 1 -- start start 2024-04-25T19:04:06.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f616d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f616813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f616d400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f616815d820 0x7f61680d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f616d400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616815dd60 con 0x7f616815d410 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f616d400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f61681401d0 con 0x7f616815d820 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f616d400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f616815e040 con 0x7f61680cdad0 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.090+0000 7f615f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f616815d410 0x7f616806f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38876/0 (socket says 172.21.15.123:38876) 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f615f400640 1 -- 172.21.15.123:0/3824206576 learned_addr learned my addr 172.21.15.123:0/3824206576 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f616ca00640 1 --2- 172.21.15.123:0/3824206576 >> v2:172.21.15.146:3300/0 conn(0x7f616815d820 0x7f61680d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167e00640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f616813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468868575 0 0) 0x7f616815dd60 con 0x7f616815d410 2024-04-25T19:04:06.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f614c003520 con 0x7f616815d410 2024-04-25T19:04:06.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1831670877 0 0) 0x7f614c003520 con 0x7f616815d410 2024-04-25T19:04:06.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 msgr2=0x7f616813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f616813fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 >> v2:172.21.15.146:3300/0 conn(0x7f616815d820 msgr2=0x7f61680d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 --2- 172.21.15.123:0/3824206576 >> v2:172.21.15.146:3300/0 conn(0x7f616815d820 0x7f61680d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61680d0280 con 0x7f616815d410 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f6150001690 con 0x7f616815d410 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.091+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 >> v1:172.21.15.123:6789/0 conn(0x7f616815d410 legacy=0x7f616806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c0039b0 0x7f614c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f614c004800 0x7f614c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f614c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61681401d0 con 0x7f614c004800 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f616815e040 con 0x7f614c0039b0 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f614c00bb50 con 0x7f61680cdad0 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f614c004800 0x7f614c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f6167e00640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c0039b0 0x7f614c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f616ca00640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f614c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 -- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 msgr2=0x7f614c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61680cdad0 0x7f614c005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 -- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c0039b0 msgr2=0x7f614c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c0039b0 0x7f614c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.092+0000 7f615f400640 1 -- 172.21.15.123:0/3824206576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c003520 con 0x7f614c004800 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.093+0000 7f615f400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f614c004800 0x7f614c004c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61680cdf70 tx=0x7f614c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b531491418a722d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150037070 con 0x7f614c004800 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61500058d0 con 0x7f614c004800 2024-04-25T19:04:06.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f6167400640 1 -- 172.21.15.123:0/3824206576 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150007ca0 con 0x7f614c004800 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 -- 172.21.15.123:0/3824206576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f614c004800 msgr2=0x7f614c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 --2- 172.21.15.123:0/3824206576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f614c004800 0x7f614c004c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61680cdf70 tx=0x7f614c005830 comp rx=0 tx=0).stop 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616ca00640 1 -- 172.21.15.123:0/3824206576 reap_dead start 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 -- 172.21.15.123:0/3824206576 shutdown_connections 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 -- 172.21.15.123:0/3824206576 >> 172.21.15.123:0/3824206576 conn(0x7f61680d19f0 msgr2=0x7f61680d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 -- 172.21.15.123:0/3824206576 shutdown_connections 2024-04-25T19:04:06.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.094+0000 7f616d400640 1 -- 172.21.15.123:0/3824206576 wait complete. 2024-04-25T19:04:06.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.097+0000 7f616d400640 1 Processor -- start 2024-04-25T19:04:06.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.097+0000 7f616d400640 1 -- start start 2024-04-25T19:04:06.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c004800 0x7f61681e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f616813cac0 0x7f616813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f616815e320 con 0x7f61681f6f80 2024-04-25T19:04:06.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f616806f770 con 0x7f614c004800 2024-04-25T19:04:06.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.098+0000 7f616d400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f616813fc90 con 0x7f616813cac0 2024-04-25T19:04:06.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42824/0 (socket says 172.21.15.123:42824) 2024-04-25T19:04:06.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 -- 172.21.15.123:0/3671354296 learned_addr learned my addr 172.21.15.123:0/3671354296 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 -- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f616813cac0 msgr2=0x7f616813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f616813cac0 0x7f616813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 -- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c004800 msgr2=0x7f61681e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c004800 0x7f61681e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 -- 172.21.15.123:0/3671354296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616813d410 con 0x7f61681f6f80 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.099+0000 7f615f400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f615000c220 tx=0x7f6150001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.100+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150037070 con 0x7f61681f6f80 2024-04-25T19:04:06.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.100+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6150038040 con 0x7f61681f6f80 2024-04-25T19:04:06.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.100+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150001be0 con 0x7f61681f6f80 2024-04-25T19:04:06.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.100+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681f3e80 con 0x7f61681f6f80 2024-04-25T19:04:06.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.100+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681f75a0 con 0x7f61681f6f80 2024-04-25T19:04:06.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.101+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168050740 con 0x7f61681f6f80 2024-04-25T19:04:06.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.102+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f615003e480 con 0x7f61681f6f80 2024-04-25T19:04:06.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.103+0000 7f6164c00640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 0x7f61440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.103+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f61500b53d0 con 0x7f61681f6f80 2024-04-25T19:04:06.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.103+0000 7f6167e00640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 0x7f61440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.104+0000 7f6167e00640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 0x7f61440769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6158009950 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.108+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f616813d410 con 0x7f61681f6f80 2024-04-25T19:04:06.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.292+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61680b48f0 con 0x7f61681f6f80 2024-04-25T19:04:06.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.293+0000 7f6164c00640 1 -- 172.21.15.123:0/3671354296 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f615007f230 con 0x7f61681f6f80 2024-04-25T19:04:06.295 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:04:06.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 msgr2=0x7f61440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 0x7f61440769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6158009950 tx=0x7f6158008040 comp rx=0 tx=0).stop 2024-04-25T19:04:06.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 msgr2=0x7f616813b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f615000c220 tx=0x7f6150001990 comp rx=0 tx=0).stop 2024-04-25T19:04:06.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 shutdown_connections 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f61440744e0 0x7f61440769f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f616813cac0 0x7f616813ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61681f6f80 0x7f616813b320 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 --2- 172.21.15.123:0/3671354296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f614c004800 0x7f61681e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 >> 172.21.15.123:0/3671354296 conn(0x7f61680d19f0 msgr2=0x7f616813f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 shutdown_connections 2024-04-25T19:04:06.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.296+0000 7f616d400640 1 -- 172.21.15.123:0/3671354296 wait complete. 2024-04-25T19:04:06.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T19:04:06.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.578+0000 7f185be00640 1 Processor -- start 2024-04-25T19:04:06.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.578+0000 7f185be00640 1 -- start start 2024-04-25T19:04:06.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c0b0ac0 0x7f185c1645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f185c160530 0x7f185c160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185be00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185c163b20 con 0x7f185c06d580 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185be00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f185c164150 con 0x7f185c160530 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f185c161060 con 0x7f185c0b0ac0 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f185c160530 0x7f185c160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f185c160530 0x7f185c160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42838/0 (socket says 172.21.15.123:42838) 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 -- 172.21.15.123:0/4072463450 learned_addr learned my addr 172.21.15.123:0/4072463450 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f185aa00640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c0b0ac0 0x7f185c1645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 -- 172.21.15.123:0/4072463450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c0b0ac0 msgr2=0x7f185c1645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c0b0ac0 0x7f185c1645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.581+0000 7f1852000640 1 -- 172.21.15.123:0/4072463450 >> v1:172.21.15.146:6789/0 conn(0x7f185c06d580 legacy=0x7f185c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.582+0000 7f1852000640 1 -- 172.21.15.123:0/4072463450 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c161340 con 0x7f185c160530 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.583+0000 7f1852000640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c160940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1844009870 tx=0x7f184402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1c23db047e2f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.583+0000 7f185a000640 1 -- 172.21.15.123:0/4072463450 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844037070 con 0x7f185c160530 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.583+0000 7f185a000640 1 -- 172.21.15.123:0/4072463450 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1844032440 con 0x7f185c160530 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.583+0000 7f185a000640 1 -- 172.21.15.123:0/4072463450 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844032810 con 0x7f185c160530 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 -- 172.21.15.123:0/4072463450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 msgr2=0x7f185c160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c160940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1844009870 tx=0x7f184402a180 comp rx=0 tx=0).stop 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 -- 172.21.15.123:0/4072463450 shutdown_connections 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c160940 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 --2- 172.21.15.123:0/4072463450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c0b0ac0 0x7f185c1645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 -- 172.21.15.123:0/4072463450 >> 172.21.15.123:0/4072463450 conn(0x7f185c073090 msgr2=0x7f185c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 -- 172.21.15.123:0/4072463450 shutdown_connections 2024-04-25T19:04:06.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.584+0000 7f185be00640 1 -- 172.21.15.123:0/4072463450 wait complete. 2024-04-25T19:04:06.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.586+0000 7f185be00640 1 Processor -- start 2024-04-25T19:04:06.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.586+0000 7f185be00640 1 -- start start 2024-04-25T19:04:06.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c06d580 0x7f185c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f185c0b0ac0 0x7f185c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f185c156890 con 0x7f185c160530 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f185c070b20 con 0x7f185c0b0ac0 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185be00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f185c156a10 con 0x7f185c06d580 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42840/0 (socket says 172.21.15.123:42840) 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 -- 172.21.15.123:0/375941569 learned_addr learned my addr 172.21.15.123:0/375941569 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f1852000640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f185c0b0ac0 0x7f185c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185aa00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c06d580 0x7f185c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 -- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c06d580 msgr2=0x7f185c15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c06d580 0x7f185c15b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 -- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f185c0b0ac0 msgr2=0x7f185c149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f185c0b0ac0 0x7f185c149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.588+0000 7f185b400640 1 -- 172.21.15.123:0/375941569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c15bb80 con 0x7f185c160530 2024-04-25T19:04:06.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f185b400640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f185400be60 tx=0x7f185400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854019070 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1854014440 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854014830 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c0ce260 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.589+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c0ce820 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.590+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c050740 con 0x7f185c160530 2024-04-25T19:04:06.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.591+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f185401c070 con 0x7f185c160530 2024-04-25T19:04:06.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.592+0000 7f1853400640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.593+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1854095560 con 0x7f185c160530 2024-04-25T19:04:06.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.593+0000 7f185aa00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.593+0000 7f185aa00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f184c00b7c0 tx=0x7f184c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.597+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f185405f8e0 con 0x7f185c160530 2024-04-25T19:04:06.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.793+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f185c0cdf90 con 0x7f185c160530 2024-04-25T19:04:06.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.794+0000 7f1853400640 1 -- 172.21.15.123:0/375941569 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f185405f1e0 con 0x7f185c160530 2024-04-25T19:04:06.796 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:04:06.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 msgr2=0x7f18380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f184c00b7c0 tx=0x7f184c008040 comp rx=0 tx=0).stop 2024-04-25T19:04:06.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 msgr2=0x7f185c0cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f185400be60 tx=0x7f185400bf50 comp rx=0 tx=0).stop 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 shutdown_connections 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185c160530 0x7f185c0cf300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f185c0b0ac0 0x7f185c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 --2- 172.21.15.123:0/375941569 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f185c06d580 0x7f185c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.797+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 >> 172.21.15.123:0/375941569 conn(0x7f185c073090 msgr2=0x7f185c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.798+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 shutdown_connections 2024-04-25T19:04:06.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:06.798+0000 7f185be00640 1 -- 172.21.15.123:0/375941569 wait complete. 2024-04-25T19:04:06.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:07.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.098+0000 7f6fb5400640 1 Processor -- start 2024-04-25T19:04:07.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.098+0000 7f6fb5400640 1 -- start start 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.100+0000 7f6fb5400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 0x7f6fb0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.100+0000 7f6fb5400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb00aec30 con 0x7f6fb0154950 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.100+0000 7f6fb5400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb00aef10 con 0x7f6fb0161b10 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.100+0000 7f6fb5400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb00d1530 con 0x7f6fb00d1120 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.101+0000 7f6faf200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6fb0154950 0x7f6fb0154d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38884/0 (socket says 172.21.15.123:38884) 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.101+0000 7f6faf200640 1 -- 172.21.15.123:0/3486148058 learned_addr learned my addr 172.21.15.123:0/3486148058 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.101+0000 7f6fae800640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 0x7f6fb0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.101+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014069153 0 0) 0x7f6fb00aec30 con 0x7f6fb0154950 2024-04-25T19:04:07.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.101+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f94003520 con 0x7f6fb0154950 2024-04-25T19:04:07.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3267687235 0 0) 0x7f6f94003520 con 0x7f6fb0154950 2024-04-25T19:04:07.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 msgr2=0x7f6fb0070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 0x7f6fb0070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 >> v1:172.21.15.146:6789/0 conn(0x7f6fb0161b10 legacy=0x7f6fb0161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb00af1f0 con 0x7f6fb0154950 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f6fa80019f0 con 0x7f6fb0154950 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 >> v1:172.21.15.123:6789/0 conn(0x7f6fb0154950 legacy=0x7f6fb0154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f940039e0 0x7f6f94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f94004800 0x7f6f94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 0x7f6f9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb00d1530 con 0x7f6f94004800 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94005590 con 0x7f6f940039e0 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9400bf00 con 0x7f6fb00d1120 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f94004800 0x7f6f94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fae800640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f940039e0 0x7f6f94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 -- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 msgr2=0x7f6f9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00d1120 0x7f6f9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 -- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f940039e0 msgr2=0x7f6f94003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f940039e0 0x7f6f94003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.102+0000 7f6fa5e00640 1 -- 172.21.15.123:0/3486148058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f94003520 con 0x7f6f94004800 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fa5e00640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f94004800 0x7f6f94004c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b1260 tx=0x7f6f94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fe2c128409ff35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98001370 con 0x7f6f94004800 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f980052d0 con 0x7f6f94004800 2024-04-25T19:04:07.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fade00640 1 -- 172.21.15.123:0/3486148058 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9802ab40 con 0x7f6f94004800 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fb5400640 1 -- 172.21.15.123:0/3486148058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f94004800 msgr2=0x7f6f94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.104+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3486148058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f94004800 0x7f6f94004c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b1260 tx=0x7f6f94005830 comp rx=0 tx=0).stop 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.105+0000 7f6faf200640 1 -- 172.21.15.123:0/3486148058 reap_dead start 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.105+0000 7f6fb5400640 1 -- 172.21.15.123:0/3486148058 shutdown_connections 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.105+0000 7f6fb5400640 1 -- 172.21.15.123:0/3486148058 >> 172.21.15.123:0/3486148058 conn(0x7f6fb00d1ae0 msgr2=0x7f6fb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.105+0000 7f6fb5400640 1 -- 172.21.15.123:0/3486148058 shutdown_connections 2024-04-25T19:04:07.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.105+0000 7f6fb5400640 1 -- 172.21.15.123:0/3486148058 wait complete. 2024-04-25T19:04:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.107+0000 7f6fb5400640 1 Processor -- start 2024-04-25T19:04:07.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.107+0000 7f6fb5400640 1 -- start start 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94004800 0x7f6fb015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb0149990 0x7f6fb01fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb00ae960 con 0x7f6fb014c810 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb00ae7e0 con 0x7f6f94004800 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fb5400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb006d8f0 con 0x7f6fb0149990 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42870/0 (socket says 172.21.15.123:42870) 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 -- 172.21.15.123:0/3354923716 learned_addr learned my addr 172.21.15.123:0/3354923716 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fa5e00640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb0149990 0x7f6fb01fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 -- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb0149990 msgr2=0x7f6fb01fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6fae800640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94004800 0x7f6fb015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb0149990 0x7f6fb01fa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 -- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94004800 msgr2=0x7f6fb015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.109+0000 7f6faf200640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94004800 0x7f6fb015b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6faf200640 1 -- 172.21.15.123:0/3354923716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb014d160 con 0x7f6fb014c810 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6faf200640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8001370 tx=0x7f6fa800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800a5b0 con 0x7f6fb014c810 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa800a770 con 0x7f6fb014c810 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800ab60 con 0x7f6fb014c810 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb01fa900 con 0x7f6fb014c810 2024-04-25T19:04:07.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.110+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb01faec0 con 0x7f6fb014c810 2024-04-25T19:04:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.112+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb01469e0 con 0x7f6fb014c810 2024-04-25T19:04:07.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.113+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fa8011070 con 0x7f6fb014c810 2024-04-25T19:04:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.113+0000 7f6fa7400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 0x7f6f8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.114+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f6fa8094c40 con 0x7f6fb014c810 2024-04-25T19:04:07.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.114+0000 7f6fae800640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 0x7f6f8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.114+0000 7f6fae800640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 0x7f6f8c0769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009910 tx=0x7f6fa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.118+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fb01469e0 con 0x7f6fb014c810 2024-04-25T19:04:07.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.358+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fb0070250 con 0x7f6fb014c810 2024-04-25T19:04:07.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.359+0000 7f6fa7400640 1 -- 172.21.15.123:0/3354923716 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f6fb014d160 con 0x7f6fb014c810 2024-04-25T19:04:07.361 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:07.361 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:04:07.362 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:04:07.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 msgr2=0x7f6f8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 0x7f6f8c0769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009910 tx=0x7f6fa0008040 comp rx=0 tx=0).stop 2024-04-25T19:04:07.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 msgr2=0x7f6fb014cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8001370 tx=0x7f6fa800ccd0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 shutdown_connections 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6f8c0744b0 0x7f6f8c0769c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fb014c810 0x7f6fb014cc20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb0149990 0x7f6fb01fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 --2- 172.21.15.123:0/3354923716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94004800 0x7f6fb015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.362+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 >> 172.21.15.123:0/3354923716 conn(0x7f6fb00d1ae0 msgr2=0x7f6fb01581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.363+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 shutdown_connections 2024-04-25T19:04:07.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.363+0000 7f6fb5400640 1 -- 172.21.15.123:0/3354923716 wait complete. 2024-04-25T19:04:07.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:07.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.658+0000 7f6372200640 1 Processor -- start 2024-04-25T19:04:07.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.658+0000 7f6372200640 1 -- start start 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f6372200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f636c06d8f0 0x7f636c154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f6372200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c071600 con 0x7f636c0d1120 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f6372200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c0aec30 con 0x7f636c161b10 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f6372200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f636c0d1530 con 0x7f636c06d8f0 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f636b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f636c0d1120 0x7f636c070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38892/0 (socket says 172.21.15.123:38892) 2024-04-25T19:04:07.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f636b400640 1 -- 172.21.15.123:0/3713852992 learned_addr learned my addr 172.21.15.123:0/3713852992 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.660+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647912328 0 0) 0x7f636c071600 con 0x7f636c0d1120 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6350003520 con 0x7f636c0d1120 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636be00640 1 --2- 172.21.15.123:0/3713852992 >> v2:172.21.15.146:3300/0 conn(0x7f636c06d8f0 0x7f636c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4219551298 0 0) 0x7f6350003520 con 0x7f636c0d1120 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 >> v2:172.21.15.146:3300/0 conn(0x7f636c06d8f0 msgr2=0x7f636c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 --2- 172.21.15.123:0/3713852992 >> v2:172.21.15.146:3300/0 conn(0x7f636c06d8f0 0x7f636c154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 >> v1:172.21.15.146:6789/0 conn(0x7f636c161b10 legacy=0x7f636c161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c0aef10 con 0x7f636c0d1120 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f635c001670 con 0x7f636c0d1120 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 >> v1:172.21.15.123:6789/0 conn(0x7f636c0d1120 legacy=0x7f636c070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63500039b0 0x7f6350003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6350004800 0x7f6350004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 0x7f635000ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0d1530 con 0x7f6350005380 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f635000c320 con 0x7f6350004800 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f635000c520 con 0x7f63500039b0 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.661+0000 7f636be00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 0x7f635000ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636b400640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63500039b0 0x7f6350003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f6362a00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6350004800 0x7f6350004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636be00640 1 -- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63500039b0 msgr2=0x7f6350003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636be00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63500039b0 0x7f6350003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636be00640 1 -- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6350004800 msgr2=0x7f6350004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636be00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6350004800 0x7f6350004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.662+0000 7f636be00640 1 -- 172.21.15.123:0/3713852992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6350003520 con 0x7f6350005380 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.663+0000 7f636be00640 1 -- 172.21.15.123:0/3713852992 reap_dead start 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.663+0000 7f636be00640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 0x7f635000ba40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f636400ba00 tx=0x7f6350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a6942afdfc9635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364030070 con 0x7f6350005380 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636402ade0 con 0x7f6350005380 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f636aa00640 1 -- 172.21.15.123:0/3713852992 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400b600 con 0x7f6350005380 2024-04-25T19:04:07.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f6372200640 1 -- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 msgr2=0x7f635000ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f6372200640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 0x7f635000ba40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f636400ba00 tx=0x7f6350005830 comp rx=0 tx=0).stop 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f6372200640 1 -- 172.21.15.123:0/3713852992 shutdown_connections 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.664+0000 7f6372200640 1 --2- 172.21.15.123:0/3713852992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6350005380 0x7f635000ba40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.665+0000 7f6372200640 1 -- 172.21.15.123:0/3713852992 >> 172.21.15.123:0/3713852992 conn(0x7f636c0d1ae0 msgr2=0x7f636c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.665+0000 7f6372200640 1 -- 172.21.15.123:0/3713852992 shutdown_connections 2024-04-25T19:04:07.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.665+0000 7f6372200640 1 -- 172.21.15.123:0/3713852992 wait complete. 2024-04-25T19:04:07.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.667+0000 7f6372200640 1 Processor -- start 2024-04-25T19:04:07.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.667+0000 7f6372200640 1 -- start start 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 0x7f636c1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 0x7f636c1f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0ae7e0 con 0x7f636c1e7bc0 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0af4d0 con 0x7f636c1f9b90 2024-04-25T19:04:07.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6372200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f636c0ae960 con 0x7f6350005380 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42884/0 (socket says 172.21.15.123:42884) 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 -- 172.21.15.123:0/678413504 learned_addr learned my addr 172.21.15.123:0/678413504 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f6362a00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 0x7f636c1f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636b400640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 0x7f636c1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 -- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 msgr2=0x7f636c1434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 0x7f636c1434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 -- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 msgr2=0x7f636c1f9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 0x7f636c1f9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.669+0000 7f636be00640 1 -- 172.21.15.123:0/678413504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1fa4e0 con 0x7f636c1e7bc0 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6362a00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 0x7f636c1f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f636b400640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 0x7f636c1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f636be00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f636400baf0 tx=0x7f63640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400b600 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6364042070 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364003e50 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c0b5260 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.670+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c0b5820 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.672+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c076a40 con 0x7f636c1e7bc0 2024-04-25T19:04:07.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.673+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6364047460 con 0x7f636c1e7bc0 2024-04-25T19:04:07.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.674+0000 7f6363e00640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 0x7f6348076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.674+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f63640b5570 con 0x7f636c1e7bc0 2024-04-25T19:04:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.674+0000 7f636b400640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 0x7f6348076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:07.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.675+0000 7f636b400640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 0x7f6348076b60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f635c001120 tx=0x7f635c001d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:07.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.679+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f636c1fa4e0 con 0x7f636c1e7bc0 2024-04-25T19:04:07.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.874+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f636c070250 con 0x7f636c1e7bc0 2024-04-25T19:04:07.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.876+0000 7f6363e00640 1 -- 172.21.15.123:0/678413504 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7f636407f8f0 con 0x7f636c1e7bc0 2024-04-25T19:04:07.878 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:07.878 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 msgr2=0x7f6348076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 0x7f6348076b60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f635c001120 tx=0x7f635c001d40 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 msgr2=0x7f636c1e7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f636400baf0 tx=0x7f63640046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 shutdown_connections 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6348074650 0x7f6348076b60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f636c1e7bc0 0x7f636c1e7fd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c1f9b90 0x7f636c1f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 --2- 172.21.15.123:0/678413504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6350005380 0x7f636c1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.878+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 >> 172.21.15.123:0/678413504 conn(0x7f636c0d1ae0 msgr2=0x7f636c0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.879+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 shutdown_connections 2024-04-25T19:04:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:07.879+0000 7f6372200640 1 -- 172.21.15.123:0/678413504 wait complete. 2024-04-25T19:04:07.905 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:04:07.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.174+0000 7fdc25c00640 1 Processor -- start 2024-04-25T19:04:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.175+0000 7fdc25c00640 1 -- start start 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc25c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc200d1120 0x7fdc20070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc25c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc200aec30 con 0x7fdc20154950 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc25c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc200aef10 con 0x7fdc20161b10 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc25c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdc200d1530 con 0x7fdc200d1120 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc1f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdc20154950 0x7fdc20154d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38900/0 (socket says 172.21.15.123:38900) 2024-04-25T19:04:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.177+0000 7fdc1f400640 1 -- 172.21.15.123:0/4002242119 learned_addr learned my addr 172.21.15.123:0/4002242119 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/4002242119 >> v2:172.21.15.146:3300/0 conn(0x7fdc200d1120 0x7fdc20070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184256221 0 0) 0x7fdc200aec30 con 0x7fdc20154950 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc04003520 con 0x7fdc20154950 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3149999196 0 0) 0x7fdc04003520 con 0x7fdc20154950 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 >> v2:172.21.15.146:3300/0 conn(0x7fdc200d1120 msgr2=0x7fdc20070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 --2- 172.21.15.123:0/4002242119 >> v2:172.21.15.146:3300/0 conn(0x7fdc200d1120 0x7fdc20070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 >> v1:172.21.15.146:6789/0 conn(0x7fdc20161b10 legacy=0x7fdc20161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc200af1f0 con 0x7fdc20154950 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fdc180019f0 con 0x7fdc20154950 2024-04-25T19:04:08.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 >> v1:172.21.15.123:6789/0 conn(0x7fdc20154950 legacy=0x7fdc20154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 0x7fdc04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc04004800 0x7fdc04004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc04005350 0x7fdc0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200d1530 con 0x7fdc040039b0 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0400c280 con 0x7fdc04005350 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0400c480 con 0x7fdc04004800 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.178+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 0x7fdc04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1f400640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc04005350 0x7fdc0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1ea00640 1 -- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc04004800 msgr2=0x7fdc04004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1e000640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc04004800 0x7fdc04004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc04004800 0x7fdc04004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1ea00640 1 -- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc04005350 msgr2=0x7fdc0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc04005350 0x7fdc0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1ea00640 1 -- 172.21.15.123:0/4002242119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc04003520 con 0x7fdc040039b0 2024-04-25T19:04:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1f400640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc04005350 0x7fdc0400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1f400640 1 -- 172.21.15.123:0/4002242119 reap_dead start 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.179+0000 7fdc1e000640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc04004800 0x7fdc04004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.180+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 0x7fdc04003e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009ac0 tx=0x7fdc04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec716bac631b80f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.180+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10036070 con 0x7fdc040039b0 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.180+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10031400 con 0x7fdc040039b0 2024-04-25T19:04:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.180+0000 7fdc1d600640 1 -- 172.21.15.123:0/4002242119 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10009690 con 0x7fdc040039b0 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 -- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 msgr2=0x7fdc04003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 0x7fdc04003e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009ac0 tx=0x7fdc04005830 comp rx=0 tx=0).stop 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 -- 172.21.15.123:0/4002242119 shutdown_connections 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 --2- 172.21.15.123:0/4002242119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc040039b0 0x7fdc04003e00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 -- 172.21.15.123:0/4002242119 >> 172.21.15.123:0/4002242119 conn(0x7fdc200d1ae0 msgr2=0x7fdc200d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 -- 172.21.15.123:0/4002242119 shutdown_connections 2024-04-25T19:04:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.181+0000 7fdc25c00640 1 -- 172.21.15.123:0/4002242119 wait complete. 2024-04-25T19:04:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.184+0000 7fdc25c00640 1 Processor -- start 2024-04-25T19:04:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.184+0000 7fdc25c00640 1 -- start start 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 0x7fdc201f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b20c0 0x7fdc200cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200ae960 con 0x7fdc201510d0 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200ae7e0 con 0x7fdc040039b0 2024-04-25T19:04:08.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc25c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdc20161f20 con 0x7fdc200b20c0 2024-04-25T19:04:08.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.185+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42898/0 (socket says 172.21.15.123:42898) 2024-04-25T19:04:08.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 -- 172.21.15.123:0/3259598796 learned_addr learned my addr 172.21.15.123:0/3259598796 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 0x7fdc201f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 -- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b20c0 msgr2=0x7fdc200cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1f400640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b20c0 0x7fdc200cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b20c0 0x7fdc200cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 -- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 msgr2=0x7fdc201f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 0x7fdc201f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 -- 172.21.15.123:0/3259598796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc200b24d0 con 0x7fdc201510d0 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 0x7fdc201f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.186+0000 7fdc1e000640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d430 tx=0x7fdc0c00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.187+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c016030 con 0x7fdc201510d0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.187+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0c007020 con 0x7fdc201510d0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.187+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007410 con 0x7fdc201510d0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.187+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc201fa3d0 con 0x7fdc201510d0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.187+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc201fa990 con 0x7fdc201510d0 2024-04-25T19:04:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.188+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc20076a40 con 0x7fdc201510d0 2024-04-25T19:04:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.189+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc0c019070 con 0x7fdc201510d0 2024-04-25T19:04:08.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.190+0000 7fdc16a00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 0x7fdc000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.190+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fdc0c0952a0 con 0x7fdc201510d0 2024-04-25T19:04:08.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.190+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 0x7fdc000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.191+0000 7fdc1ea00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 0x7fdc000769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc10033000 tx=0x7fdc10008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.195+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc200b24d0 con 0x7fdc201510d0 2024-04-25T19:04:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.439+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc20070250 con 0x7fdc201510d0 2024-04-25T19:04:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.440+0000 7fdc16a00640 1 -- 172.21.15.123:0/3259598796 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fdc0c05f620 con 0x7fdc201510d0 2024-04-25T19:04:08.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:08.443 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:04:08.443 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:04:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 msgr2=0x7fdc000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 0x7fdc000769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc10033000 tx=0x7fdc10008030 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 msgr2=0x7fdc200b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d430 tx=0x7fdc0c00d8f0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 shutdown_connections 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fdc000744b0 0x7fdc000769c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b20c0 0x7fdc200cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc201510d0 0x7fdc200b1b80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 --2- 172.21.15.123:0/3259598796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc040039b0 0x7fdc201f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.443+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 >> 172.21.15.123:0/3259598796 conn(0x7fdc200d1ae0 msgr2=0x7fdc200cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.444+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 shutdown_connections 2024-04-25T19:04:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.444+0000 7fdc25c00640 1 -- 172.21.15.123:0/3259598796 wait complete. 2024-04-25T19:04:08.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.741+0000 7f69c7200640 1 Processor -- start 2024-04-25T19:04:08.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.741+0000 7f69c7200640 1 -- start start 2024-04-25T19:04:08.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.743+0000 7f69c7200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69c006fe20 0x7f69c015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 0x7f69c015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c7200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c015f070 con 0x7f69c006d580 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c7200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f69c006f340 con 0x7f69c006fe20 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69c006f620 con 0x7f69c015e2f0 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c5400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f69c006d580 0x7f69c006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38906/0 (socket says 172.21.15.123:38906) 2024-04-25T19:04:08.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c5400640 1 -- 172.21.15.123:0/170175417 learned_addr learned my addr 172.21.15.123:0/170175417 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c6800640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 0x7f69c015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c5e00640 1 --2- 172.21.15.123:0/170175417 >> v2:172.21.15.146:3300/0 conn(0x7f69c006fe20 0x7f69c015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.744+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168252802 0 0) 0x7f69c015f070 con 0x7f69c006d580 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69a8003520 con 0x7f69c006d580 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1578434143 0 0) 0x7f69a8003520 con 0x7f69c006d580 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 msgr2=0x7f69c015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 0x7f69c015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 >> v2:172.21.15.146:3300/0 conn(0x7f69c006fe20 msgr2=0x7f69c015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 --2- 172.21.15.123:0/170175417 >> v2:172.21.15.146:3300/0 conn(0x7f69c006fe20 0x7f69c015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c006f900 con 0x7f69c006d580 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f69b0001670 con 0x7f69c006d580 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 >> v1:172.21.15.123:6789/0 conn(0x7f69c006d580 legacy=0x7f69c006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a80039b0 0x7f69a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 0x7f69a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a8004d40 0x7f69a8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69c006f340 con 0x7f69a80039b0 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69c006f620 con 0x7f69a8004d40 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69a800ba40 con 0x7f69c015e2f0 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.745+0000 7f69c5e00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a80039b0 0x7f69a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.746+0000 7f69c5e00640 1 -- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 msgr2=0x7f69a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.746+0000 7f69c5e00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c015e2f0 0x7f69a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.746+0000 7f69c5e00640 1 -- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a8004d40 msgr2=0x7f69a8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.746+0000 7f69c5e00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a8004d40 0x7f69a8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.746+0000 7f69c5e00640 1 -- 172.21.15.123:0/170175417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8003520 con 0x7f69a80039b0 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.747+0000 7f69c5e00640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a80039b0 0x7f69a8003e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69c0070d80 tx=0x7f69a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d279c83528c40b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.747+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4036070 con 0x7f69a80039b0 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.747+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b4031400 con 0x7f69a80039b0 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.747+0000 7f69c4a00640 1 -- 172.21.15.123:0/170175417 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4009690 con 0x7f69a80039b0 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 -- 172.21.15.123:0/170175417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a80039b0 msgr2=0x7f69a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 --2- 172.21.15.123:0/170175417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a80039b0 0x7f69a8003e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69c0070d80 tx=0x7f69a8005830 comp rx=0 tx=0).stop 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c6800640 1 -- 172.21.15.123:0/170175417 reap_dead start 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 -- 172.21.15.123:0/170175417 shutdown_connections 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 -- 172.21.15.123:0/170175417 >> 172.21.15.123:0/170175417 conn(0x7f69c0073200 msgr2=0x7f69c0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 -- 172.21.15.123:0/170175417 shutdown_connections 2024-04-25T19:04:08.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.748+0000 7f69c7200640 1 -- 172.21.15.123:0/170175417 wait complete. 2024-04-25T19:04:08.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.750+0000 7f69c7200640 1 Processor -- start 2024-04-25T19:04:08.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.751+0000 7f69c7200640 1 -- start start 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a80039b0 0x7f69c013e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c01e7700 0x7f69c013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69c015e880 con 0x7f69c01e97c0 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69c015e700 con 0x7f69a80039b0 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69c015ea00 con 0x7f69c01e7700 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42920/0 (socket says 172.21.15.123:42920) 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.752+0000 7f69c6800640 1 -- 172.21.15.123:0/2129254174 learned_addr learned my addr 172.21.15.123:0/2129254174 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c5e00640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a80039b0 0x7f69c013e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 -- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c01e7700 msgr2=0x7f69c013fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c01e7700 0x7f69c013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 -- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a80039b0 msgr2=0x7f69c013e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:08.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a80039b0 0x7f69c013e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:08.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 -- 172.21.15.123:0/2129254174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01ea110 con 0x7f69c01e97c0 2024-04-25T19:04:08.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69c6800640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f69c0070230 tx=0x7f69bc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00bb10 con 0x7f69c01e97c0 2024-04-25T19:04:08.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69bc01a070 con 0x7f69c01e97c0 2024-04-25T19:04:08.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.753+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc014400 con 0x7f69c01e97c0 2024-04-25T19:04:08.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.754+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01e8be0 con 0x7f69c01e97c0 2024-04-25T19:04:08.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.754+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c0144c90 con 0x7f69c01e97c0 2024-04-25T19:04:08.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.755+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c0076a40 con 0x7f69c01e97c0 2024-04-25T19:04:08.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.756+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69bc005d10 con 0x7f69c01e97c0 2024-04-25T19:04:08.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.757+0000 7f69ba000640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 0x7f69a40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:08.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.758+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f69bc095610 con 0x7f69c01e97c0 2024-04-25T19:04:08.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.758+0000 7f69c5e00640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 0x7f69a40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:08.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.758+0000 7f69c5e00640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 0x7f69a40769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:08.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.763+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69c0076a40 con 0x7f69c01e97c0 2024-04-25T19:04:08.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.997+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69c015edb0 con 0x7f69c01e97c0 2024-04-25T19:04:09.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:08.998+0000 7f69ba000640 1 -- 172.21.15.123:0/2129254174 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f69c01ea110 con 0x7f69c01e97c0 2024-04-25T19:04:09.001 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:09.001 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1909232432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1909232432},{"type":"v1","addr":"172.21.15.146:6833","nonce":1909232432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/84685581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":84685581},{"type":"v1","addr":"172.21.15.123:6836","nonce":84685581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2353614717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2353614717},{"type":"v1","addr":"172.21.15.123:6837","nonce":2353614717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:02:39.551680+0000","modified":"2024-04-25T19:03:03.252379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/1141870368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1141870368},{"type":"v1","addr":"172.21.15.146:6835","nonce":1141870368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-04-25T19:04:09.001 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-25T19:04:09.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 msgr2=0x7f69a40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 0x7f69a40769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b4008030 comp rx=0 tx=0).stop 2024-04-25T19:04:09.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 msgr2=0x7f69c01e9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f69c0070230 tx=0x7f69bc00b480 comp rx=0 tx=0).stop 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 shutdown_connections 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f69a40744e0 0x7f69a40769f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c01e97c0 0x7f69c01e9bd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69c01e7700 0x7f69c013fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 --2- 172.21.15.123:0/2129254174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a80039b0 0x7f69c013e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 >> 172.21.15.123:0/2129254174 conn(0x7f69c0073200 msgr2=0x7f69c0160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 shutdown_connections 2024-04-25T19:04:09.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.002+0000 7f69c7200640 1 -- 172.21.15.123:0/2129254174 wait complete. 2024-04-25T19:04:09.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:09.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.286+0000 7fc42dc00640 1 Processor -- start 2024-04-25T19:04:09.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.286+0000 7fc42dc00640 1 -- start start 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc42dc00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc42806d580 0x7fc42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc42dc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc42806ded0 con 0x7fc42806fe20 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc42dc00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc42815f070 con 0x7fc42815e2f0 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc42dc00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc42815ee10 con 0x7fc42806d580 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc427e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc42806fe20 0x7fc42815b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38914/0 (socket says 172.21.15.123:38914) 2024-04-25T19:04:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.288+0000 7fc427e00640 1 -- 172.21.15.123:0/631640464 learned_addr learned my addr 172.21.15.123:0/631640464 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:09.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc427400640 1 --2- 172.21.15.123:0/631640464 >> v2:172.21.15.146:3300/0 conn(0x7fc42806d580 0x7fc42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685735106 0 0) 0x7fc418001110 con 0x7fc42806fe20 2024-04-25T19:04:09.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc40c003520 con 0x7fc42806fe20 2024-04-25T19:04:09.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574912153 0 0) 0x7fc40c003520 con 0x7fc42806fe20 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 >> v2:172.21.15.146:3300/0 conn(0x7fc42806d580 msgr2=0x7fc42806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 --2- 172.21.15.123:0/631640464 >> v2:172.21.15.146:3300/0 conn(0x7fc42806d580 0x7fc42806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 >> v1:172.21.15.146:6789/0 conn(0x7fc42815e2f0 legacy=0x7fc42815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42806f340 con 0x7fc42806fe20 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fc418001840 con 0x7fc42806fe20 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 >> v1:172.21.15.123:6789/0 conn(0x7fc42806fe20 legacy=0x7fc42815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c0039b0 0x7fc40c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 0x7fc40c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc40c005250 0x7fc40c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc42815ee10 con 0x7fc40c004800 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c00c1e0 con 0x7fc40c0039b0 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.289+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c00c3e0 con 0x7fc40c005250 2024-04-25T19:04:09.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.290+0000 7fc427400640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 0x7fc40c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.290+0000 7fc427e00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c0039b0 0x7fc40c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc427400640 1 -- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc40c005250 msgr2=0x7fc40c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc427400640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc40c005250 0x7fc40c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc427400640 1 -- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c0039b0 msgr2=0x7fc40c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc427400640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c0039b0 0x7fc40c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc427400640 1 -- 172.21.15.123:0/631640464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c003520 con 0x7fc40c004800 2024-04-25T19:04:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.291+0000 7fc42d200640 1 -- 172.21.15.123:0/631640464 reap_dead start 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.292+0000 7fc427400640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 0x7fc40c004c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc410009ac0 tx=0x7fc40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d05b1244d6255e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.292+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410036070 con 0x7fc40c004800 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.292+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc410031400 con 0x7fc40c004800 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.292+0000 7fc426a00640 1 -- 172.21.15.123:0/631640464 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410009690 con 0x7fc40c004800 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 -- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 msgr2=0x7fc40c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 0x7fc40c004c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc410009ac0 tx=0x7fc40c005830 comp rx=0 tx=0).stop 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 -- 172.21.15.123:0/631640464 shutdown_connections 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 --2- 172.21.15.123:0/631640464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c004800 0x7fc40c004c10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 -- 172.21.15.123:0/631640464 >> 172.21.15.123:0/631640464 conn(0x7fc428073200 msgr2=0x7fc428073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:09.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 -- 172.21.15.123:0/631640464 shutdown_connections 2024-04-25T19:04:09.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.293+0000 7fc42dc00640 1 -- 172.21.15.123:0/631640464 wait complete. 2024-04-25T19:04:09.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.295+0000 7fc42dc00640 1 Processor -- start 2024-04-25T19:04:09.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.295+0000 7fc42dc00640 1 -- start start 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c004800 0x7fc4281fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4280b5920 0x7fc4280b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc42806d990 con 0x7fc4281ebe10 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc42806f900 con 0x7fc40c004800 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42dc00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc42806db10 con 0x7fc4280b5920 2024-04-25T19:04:09.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42950/0 (socket says 172.21.15.123:42950) 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 -- 172.21.15.123:0/251355753 learned_addr learned my addr 172.21.15.123:0/251355753 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 -- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4280b5920 msgr2=0x7fc4280b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc42d200640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4280b5920 0x7fc4280b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4280b5920 0x7fc4280b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 -- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c004800 msgr2=0x7fc4281fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c004800 0x7fc4281fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.297+0000 7fc427400640 1 -- 172.21.15.123:0/251355753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4280b6270 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc427400640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc410009be0 tx=0x7fc410004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41003f020 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc410037040 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410036070 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc428137890 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.298+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281eba50 con 0x7fc4281ebe10 2024-04-25T19:04:09.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.299+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc428050740 con 0x7fc4281ebe10 2024-04-25T19:04:09.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.301+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc410042020 con 0x7fc4281ebe10 2024-04-25T19:04:09.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.301+0000 7fc41fe00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 0x7fc4040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.302+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc4100b5480 con 0x7fc4281ebe10 2024-04-25T19:04:09.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.302+0000 7fc427e00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 0x7fc4040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.303+0000 7fc427e00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 0x7fc4040769f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc418001eb0 tx=0x7fc418000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.306+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc410049070 con 0x7fc4281ebe10 2024-04-25T19:04:09.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.491+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc428070a00 con 0x7fc4281ebe10 2024-04-25T19:04:09.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.493+0000 7fc41fe00640 1 -- 172.21.15.123:0/251355753 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18660 (secure 0 0 0) 0x7fc4280b6270 con 0x7fc4281ebe10 2024-04-25T19:04:09.495 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:09.495 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:03:20.120597+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:02:37.807902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:02:38.536776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T19:02:42.111213+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.495+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 msgr2=0x7fc4040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.495+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 0x7fc4040769f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc418001eb0 tx=0x7fc418000ed0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.495+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 msgr2=0x7fc4281ec220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.495+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc410009be0 tx=0x7fc410004730 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 shutdown_connections 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fc4040744e0 0x7fc4040769f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4280b5920 0x7fc4280b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4281ebe10 0x7fc4281ec220 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 --2- 172.21.15.123:0/251355753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc40c004800 0x7fc4281fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 >> 172.21.15.123:0/251355753 conn(0x7fc428073200 msgr2=0x7fc428073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 shutdown_connections 2024-04-25T19:04:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.496+0000 7fc42dc00640 1 -- 172.21.15.123:0/251355753 wait complete. 2024-04-25T19:04:09.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T19:04:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.778+0000 7fbd92400640 1 Processor -- start 2024-04-25T19:04:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.779+0000 7fbd92400640 1 -- start start 2024-04-25T19:04:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd92400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd8c06fe20 0x7fbd8c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd92400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd8c15e2f0 0x7fbd8c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd92400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8c15f070 con 0x7fbd8c06d580 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd92400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c06f340 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd92400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c06f620 con 0x7fbd8c06fe20 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd91a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd8c15e2f0 0x7fbd8c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd91a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd8c15e2f0 0x7fbd8c15e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42956/0 (socket says 172.21.15.123:42956) 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.781+0000 7fbd91a00640 1 -- 172.21.15.123:0/1270500666 learned_addr learned my addr 172.21.15.123:0/1270500666 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.782+0000 7fbd91000640 1 --2- 172.21.15.123:0/1270500666 >> v2:172.21.15.146:3300/0 conn(0x7fbd8c06fe20 0x7fbd8c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.782+0000 7fbd91a00640 1 -- 172.21.15.123:0/1270500666 >> v2:172.21.15.146:3300/0 conn(0x7fbd8c06fe20 msgr2=0x7fbd8c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.782+0000 7fbd91a00640 1 --2- 172.21.15.123:0/1270500666 >> v2:172.21.15.146:3300/0 conn(0x7fbd8c06fe20 0x7fbd8c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.782+0000 7fbd91a00640 1 -- 172.21.15.123:0/1270500666 >> v1:172.21.15.146:6789/0 conn(0x7fbd8c06d580 legacy=0x7fbd8c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.782+0000 7fbd91a00640 1 -- 172.21.15.123:0/1270500666 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c06f900 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.783+0000 7fbd91a00640 1 --2- 172.21.15.123:0/1270500666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15e700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c074310 tx=0x7fbd8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c357648a2fd01b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.783+0000 7fbd8be00640 1 -- 172.21.15.123:0/1270500666 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84036070 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.783+0000 7fbd8be00640 1 -- 172.21.15.123:0/1270500666 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd84031600 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.783+0000 7fbd8be00640 1 -- 172.21.15.123:0/1270500666 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd840319d0 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 -- 172.21.15.123:0/1270500666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c15e2f0 msgr2=0x7fbd8c15e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 --2- 172.21.15.123:0/1270500666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15e700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c074310 tx=0x7fbd8402a660 comp rx=0 tx=0).stop 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 -- 172.21.15.123:0/1270500666 shutdown_connections 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 --2- 172.21.15.123:0/1270500666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15e700 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 --2- 172.21.15.123:0/1270500666 >> v2:172.21.15.146:3300/0 conn(0x7fbd8c06fe20 0x7fbd8c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 -- 172.21.15.123:0/1270500666 >> 172.21.15.123:0/1270500666 conn(0x7fbd8c073200 msgr2=0x7fbd8c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 -- 172.21.15.123:0/1270500666 shutdown_connections 2024-04-25T19:04:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.784+0000 7fbd92400640 1 -- 172.21.15.123:0/1270500666 wait complete. 2024-04-25T19:04:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.787+0000 7fbd92400640 1 Processor -- start 2024-04-25T19:04:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.787+0000 7fbd92400640 1 -- start start 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8c06d580 0x7fbd8c1f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c15edc0 con 0x7fbd8c06fe20 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c15ec40 con 0x7fbd8c15e2f0 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd92400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c0ce500 con 0x7fbd8c06d580 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42970/0 (socket says 172.21.15.123:42970) 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.789+0000 7fbd83e00640 1 -- 172.21.15.123:0/604487449 learned_addr learned my addr 172.21.15.123:0/604487449 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd91000640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8c06d580 0x7fbd8c1f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 -- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8c06d580 msgr2=0x7fbd8c1f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8c06d580 0x7fbd8c1f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 -- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c15e2f0 msgr2=0x7fbd8c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 -- 172.21.15.123:0/604487449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c144310 con 0x7fbd8c06fe20 2024-04-25T19:04:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd83e00640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400b3e0 tx=0x7fbd7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74015020 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd74011850 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.790+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c1445f0 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.791+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74011c40 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.791+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c144bb0 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.792+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c1e8ff0 con 0x7fbd8c06fe20 2024-04-25T19:04:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.793+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd74018020 con 0x7fbd8c06fe20 2024-04-25T19:04:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.794+0000 7fbd89600640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 0x7fbd680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.794+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fbd74095580 con 0x7fbd8c06fe20 2024-04-25T19:04:09.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.794+0000 7fbd91000640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 0x7fbd680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:09.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.795+0000 7fbd91000640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 0x7fbd680769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0742c0 tx=0x7fbd7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:09.799+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd7401f020 con 0x7fbd8c06fe20 2024-04-25T19:04:10.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.036+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbd8c15b740 con 0x7fbd8c06fe20 2024-04-25T19:04:10.040 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-04-25T19:04:10.038+0000 7fed86c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:04:10.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.042+0000 7fbd89600640 1 -- 172.21.15.123:0/604487449 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fbd7405f990 con 0x7fbd8c06fe20 2024-04-25T19:04:10.044 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.045+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 msgr2=0x7fbd680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.045+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 0x7fbd680769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0742c0 tx=0x7fbd7c008040 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.045+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 msgr2=0x7fbd8c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.045+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400b3e0 tx=0x7fbd7400b8a0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 shutdown_connections 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd680744b0 0x7fbd680769c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c15e2f0 0x7fbd8c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd8c06fe20 0x7fbd8c15a1a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 --2- 172.21.15.123:0/604487449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8c06d580 0x7fbd8c1f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 >> 172.21.15.123:0/604487449 conn(0x7fbd8c073200 msgr2=0x7fbd8c160b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:10.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 shutdown_connections 2024-04-25T19:04:10.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.046+0000 7fbd92400640 1 -- 172.21.15.123:0/604487449 wait complete. 2024-04-25T19:04:10.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T19:04:10.220 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-04-25T19:04:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.337+0000 7f9588000640 1 Processor -- start 2024-04-25T19:04:10.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.337+0000 7f9588000640 1 -- start start 2024-04-25T19:04:10.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.339+0000 7f9588000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 0x7f958015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.339+0000 7f9588000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95800cd140 0x7f95800af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.339+0000 7f9588000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95800cc5f0 con 0x7f9580162540 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.339+0000 7f9588000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f958006deb0 con 0x7f95800cd140 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.339+0000 7f9588000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f958006d610 con 0x7f958006daa0 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95800cd140 0x7f95800af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f95800cd140 0x7f95800af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42976/0 (socket says 172.21.15.123:42976) 2024-04-25T19:04:10.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 -- 172.21.15.123:0/3366000751 learned_addr learned my addr 172.21.15.123:0/3366000751 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 -- 172.21.15.123:0/3366000751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 msgr2=0x7f958015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9585600640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 0x7f958015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 0x7f958015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 -- 172.21.15.123:0/3366000751 >> v1:172.21.15.146:6789/0 conn(0x7f9580162540 legacy=0x7f95800cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9586000640 1 -- 172.21.15.123:0/3366000751 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958006ca00 con 0x7f95800cd140 2024-04-25T19:04:10.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.340+0000 7f9585600640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 0x7f958015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:10.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.341+0000 7f9586000640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95800cd140 0x7f95800af040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3239a47103d0127b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:10.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.342+0000 7f957fe00640 1 -- 172.21.15.123:0/3366000751 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578036070 con 0x7f95800cd140 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.342+0000 7f957fe00640 1 -- 172.21.15.123:0/3366000751 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9578031600 con 0x7f95800cd140 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.342+0000 7f957fe00640 1 -- 172.21.15.123:0/3366000751 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780319d0 con 0x7f95800cd140 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 -- 172.21.15.123:0/3366000751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95800cd140 msgr2=0x7f95800af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95800cd140 0x7f95800af040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a6d0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 -- 172.21.15.123:0/3366000751 shutdown_connections 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95800cd140 0x7f95800af040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 --2- 172.21.15.123:0/3366000751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958006daa0 0x7f958015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 -- 172.21.15.123:0/3366000751 >> 172.21.15.123:0/3366000751 conn(0x7f95800ce390 msgr2=0x7f95800ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 -- 172.21.15.123:0/3366000751 shutdown_connections 2024-04-25T19:04:10.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.343+0000 7f9588000640 1 -- 172.21.15.123:0/3366000751 wait complete. 2024-04-25T19:04:10.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.345+0000 7f9588000640 1 Processor -- start 2024-04-25T19:04:10.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.345+0000 7f9588000640 1 -- start start 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958006daa0 0x7f958009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9580162540 0x7f95800c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f958006d2a0 con 0x7f95801532f0 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f958006cfc0 con 0x7f958006daa0 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9588000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9580162950 con 0x7f9580162540 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42986/0 (socket says 172.21.15.123:42986) 2024-04-25T19:04:10.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 -- 172.21.15.123:0/2094696352 learned_addr learned my addr 172.21.15.123:0/2094696352 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:10.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 -- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9580162540 msgr2=0x7f95800c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:10.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9580162540 0x7f95800c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 -- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958006daa0 msgr2=0x7f958009d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:10.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958006daa0 0x7f958009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:10.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.347+0000 7f9586000640 1 -- 172.21.15.123:0/2094696352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958009dfa0 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f9586000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f9578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578036050 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9578042070 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578009510 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958009e280 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.348+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958009e840 con 0x7f95801532f0 2024-04-25T19:04:10.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.349+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9580076a40 con 0x7f95801532f0 2024-04-25T19:04:10.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.350+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9578007bb0 con 0x7f95801532f0 2024-04-25T19:04:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.351+0000 7f957d600640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 0x7f95600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.351+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6676+0+0 (secure 0 0 0) 0x7f95780b66e0 con 0x7f95801532f0 2024-04-25T19:04:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.351+0000 7f9585600640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 0x7f95600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.352+0000 7f9585600640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 0x7f95600769f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9574006fd0 tx=0x7f9574008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.356+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9578080a00 con 0x7f95801532f0 2024-04-25T19:04:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:10.595+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f958009dd30 con 0x7f95801532f0 2024-04-25T19:04:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.046+0000 7f957d600640 1 -- 172.21.15.123:0/2094696352 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f9578080300 con 0x7f95801532f0 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 msgr2=0x7f95600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 0x7f95600769f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9574006fd0 tx=0x7f9574008040 comp rx=0 tx=0).stop 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 msgr2=0x7f9580160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f9578004a40 comp rx=0 tx=0).stop 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 shutdown_connections 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f95600744e0 0x7f95600769f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95801532f0 0x7f9580160290 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9580162540 0x7f95800c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 --2- 172.21.15.123:0/2094696352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958006daa0 0x7f958009d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 >> 172.21.15.123:0/2094696352 conn(0x7f95800ce390 msgr2=0x7f9580161da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.049+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 shutdown_connections 2024-04-25T19:04:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.050+0000 7f9588000640 1 -- 172.21.15.123:0/2094696352 wait complete. 2024-04-25T19:04:11.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T19:04:11.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.342+0000 7f08dd800640 1 Processor -- start 2024-04-25T19:04:11.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.342+0000 7f08dd800640 1 -- start start 2024-04-25T19:04:11.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08dd800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08d8143240 0x7f08d80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08dd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d80b6640 con 0x7f08d8146090 2024-04-25T19:04:11.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08dd800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d80b68e0 con 0x7f08d806d580 2024-04-25T19:04:11.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08dd800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f08d8143650 con 0x7f08d8143240 2024-04-25T19:04:11.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08d7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f08d8146090 0x7f08d8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38922/0 (socket says 172.21.15.123:38922) 2024-04-25T19:04:11.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.344+0000 7f08d7e00640 1 -- 172.21.15.123:0/316383399 learned_addr learned my addr 172.21.15.123:0/316383399 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08cf400640 1 --2- 172.21.15.123:0/316383399 >> v2:172.21.15.146:3300/0 conn(0x7f08d8143240 0x7f08d80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013119484 0 0) 0x7f08d80b6640 con 0x7f08d8146090 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08bc005730 con 0x7f08d8146090 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4000166706 0 0) 0x7f08bc005730 con 0x7f08d8146090 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 >> v2:172.21.15.146:3300/0 conn(0x7f08d8143240 msgr2=0x7f08d80b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 --2- 172.21.15.123:0/316383399 >> v2:172.21.15.146:3300/0 conn(0x7f08d8143240 0x7f08d80b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 >> v1:172.21.15.146:6789/0 conn(0x7f08d806d580 legacy=0x7f08d806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d80b6bc0 con 0x7f08d8146090 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.345+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f08c8001670 con 0x7f08d8146090 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 >> v1:172.21.15.123:6789/0 conn(0x7f08d8146090 legacy=0x7f08d8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08bc005c10 0x7f08bc006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc006a10 0x7f08bc006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 0x7f08bc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8143650 con 0x7f08bc007480 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc00e3a0 con 0x7f08bc006a10 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f08bc00e5a0 con 0x7f08bc005c10 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 0x7f08bc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 -- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08bc005c10 msgr2=0x7f08bc006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08bc005c10 0x7f08bc006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 -- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc006a10 msgr2=0x7f08bc006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc006a10 0x7f08bc006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.346+0000 7f08dce00640 1 -- 172.21.15.123:0/316383399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc005730 con 0x7f08bc007480 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.347+0000 7f08dce00640 1 -- 172.21.15.123:0/316383399 reap_dead start 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.347+0000 7f08dce00640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 0x7f08bc00dbd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001e50 tx=0x7f08bc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cd5026b15fea00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:11.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.347+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0001a70 con 0x7f08bc007480 2024-04-25T19:04:11.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.347+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0003970 con 0x7f08bc007480 2024-04-25T19:04:11.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.347+0000 7f08d7400640 1 -- 172.21.15.123:0/316383399 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0003d60 con 0x7f08bc007480 2024-04-25T19:04:11.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 -- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 msgr2=0x7f08bc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 0x7f08bc00dbd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001e50 tx=0x7f08bc007a40 comp rx=0 tx=0).stop 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 -- 172.21.15.123:0/316383399 shutdown_connections 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 --2- 172.21.15.123:0/316383399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08bc007480 0x7f08bc00dbd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 -- 172.21.15.123:0/316383399 >> 172.21.15.123:0/316383399 conn(0x7f08d8072fb0 msgr2=0x7f08d80733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 -- 172.21.15.123:0/316383399 shutdown_connections 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.348+0000 7f08dd800640 1 -- 172.21.15.123:0/316383399 wait complete. 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.351+0000 7f08dd800640 1 Processor -- start 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.351+0000 7f08dd800640 1 -- start start 2024-04-25T19:04:11.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 0x7f08d80c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08d80afcd0 0x7f08d80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80b73c0 con 0x7f08d815f970 2024-04-25T19:04:11.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80b6f30 con 0x7f08bc007480 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08dd800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f08d80b7540 con 0x7f08d80afcd0 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43012/0 (socket says 172.21.15.123:43012) 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.352+0000 7f08cf400640 1 -- 172.21.15.123:0/3269174445 learned_addr learned my addr 172.21.15.123:0/3269174445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08d7e00640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 0x7f08d80c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 -- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08d80afcd0 msgr2=0x7f08d80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08d80afcd0 0x7f08d80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 -- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 msgr2=0x7f08d80c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 0x7f08d80c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 -- 172.21.15.123:0/3269174445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d80b0620 con 0x7f08d815f970 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08d7e00640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 0x7f08d80c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08cf400640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e7a0 tx=0x7f08c000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0017070 con 0x7f08d815f970 2024-04-25T19:04:11.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c000c940 con 0x7f08d815f970 2024-04-25T19:04:11.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d80c78b0 con 0x7f08d815f970 2024-04-25T19:04:11.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.353+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000cd10 con 0x7f08d815f970 2024-04-25T19:04:11.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.354+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d80c7e70 con 0x7f08d815f970 2024-04-25T19:04:11.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.355+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d80c7620 con 0x7f08d815f970 2024-04-25T19:04:11.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.356+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08c001a070 con 0x7f08d815f970 2024-04-25T19:04:11.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.357+0000 7f08d4c00640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 0x7f08b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.357+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6676+0+0 (secure 0 0 0) 0x7f08c0095490 con 0x7f08d815f970 2024-04-25T19:04:11.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.357+0000 7f08d7e00640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 0x7f08b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.358+0000 7f08d7e00640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 0x7f08b40769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08c80079f0 tx=0x7f08c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:11.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.363+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08d80b0620 con 0x7f08d815f970 2024-04-25T19:04:11.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:11.565+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08d8073ee0 con 0x7f08d815f970 2024-04-25T19:04:12.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.345+0000 7f08d4c00640 1 -- 172.21.15.123:0/3269174445 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f08c005f840 con 0x7f08d815f970 2024-04-25T19:04:12.348 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-04-25T19:04:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.350+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 msgr2=0x7f08b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.350+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 0x7f08b40769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08c80079f0 tx=0x7f08c8009040 comp rx=0 tx=0).stop 2024-04-25T19:04:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.350+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 msgr2=0x7f08d8141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.350+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e7a0 tx=0x7f08c000ec60 comp rx=0 tx=0).stop 2024-04-25T19:04:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 shutdown_connections 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f08b40744e0 0x7f08b40769f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08d80afcd0 0x7f08d80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d815f970 0x7f08d8141e90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 --2- 172.21.15.123:0/3269174445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc007480 0x7f08d80c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.351+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 >> 172.21.15.123:0/3269174445 conn(0x7f08d8072fb0 msgr2=0x7f08d80b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.352+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 shutdown_connections 2024-04-25T19:04:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.352+0000 7f08dd800640 1 -- 172.21.15.123:0/3269174445 wait complete. 2024-04-25T19:04:12.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T19:04:12.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.656+0000 7fed86c00640 1 Processor -- start 2024-04-25T19:04:12.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.656+0000 7fed86c00640 1 -- start start 2024-04-25T19:04:12.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed8015d330 0x7fed800b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed8006d580 0x7fed8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8006ded0 con 0x7fed80070ac0 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fed800b6550 con 0x7fed8006d580 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fed800b5040 con 0x7fed8015d330 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed8006d580 0x7fed8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed8006d580 0x7fed8006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43026/0 (socket says 172.21.15.123:43026) 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.658+0000 7fed86200640 1 -- 172.21.15.123:0/4184298936 learned_addr learned my addr 172.21.15.123:0/4184298936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.659+0000 7fed84e00640 1 --2- 172.21.15.123:0/4184298936 >> v2:172.21.15.146:3300/0 conn(0x7fed8015d330 0x7fed800b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.659+0000 7fed86200640 1 -- 172.21.15.123:0/4184298936 >> v2:172.21.15.146:3300/0 conn(0x7fed8015d330 msgr2=0x7fed800b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.659+0000 7fed86200640 1 --2- 172.21.15.123:0/4184298936 >> v2:172.21.15.146:3300/0 conn(0x7fed8015d330 0x7fed800b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.659+0000 7fed86200640 1 -- 172.21.15.123:0/4184298936 >> v1:172.21.15.146:6789/0 conn(0x7fed80070ac0 legacy=0x7fed800cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:12.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.659+0000 7fed86200640 1 -- 172.21.15.123:0/4184298936 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8015e320 con 0x7fed8006d580 2024-04-25T19:04:12.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.660+0000 7fed86200640 1 --2- 172.21.15.123:0/4184298936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed8006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fed800b5460 tx=0x7fed7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14b439ee35efe13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:12.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.660+0000 7fed7fe00640 1 -- 172.21.15.123:0/4184298936 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78037070 con 0x7fed8006d580 2024-04-25T19:04:12.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.660+0000 7fed7fe00640 1 -- 172.21.15.123:0/4184298936 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed78032440 con 0x7fed8006d580 2024-04-25T19:04:12.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.660+0000 7fed7fe00640 1 -- 172.21.15.123:0/4184298936 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78032810 con 0x7fed8006d580 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 -- 172.21.15.123:0/4184298936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 msgr2=0x7fed8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 --2- 172.21.15.123:0/4184298936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed8006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fed800b5460 tx=0x7fed7802a230 comp rx=0 tx=0).stop 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 -- 172.21.15.123:0/4184298936 shutdown_connections 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 --2- 172.21.15.123:0/4184298936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed8006d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 --2- 172.21.15.123:0/4184298936 >> v2:172.21.15.146:3300/0 conn(0x7fed8015d330 0x7fed800b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 -- 172.21.15.123:0/4184298936 >> 172.21.15.123:0/4184298936 conn(0x7fed80072fd0 msgr2=0x7fed800733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 -- 172.21.15.123:0/4184298936 shutdown_connections 2024-04-25T19:04:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.661+0000 7fed86c00640 1 -- 172.21.15.123:0/4184298936 wait complete. 2024-04-25T19:04:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.664+0000 7fed86c00640 1 Processor -- start 2024-04-25T19:04:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.664+0000 7fed86c00640 1 -- start start 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed80070ac0 0x7fed801e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed8015d330 0x7fed800b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed8015dd60 con 0x7fed8006d580 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed800b6810 con 0x7fed8015d330 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed86c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed8015e040 con 0x7fed80070ac0 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.665+0000 7fed85800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43032/0 (socket says 172.21.15.123:43032) 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 -- 172.21.15.123:0/1399670017 learned_addr learned my addr 172.21.15.123:0/1399670017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed86200640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed8015d330 0x7fed800b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:12.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 -- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed80070ac0 msgr2=0x7fed801e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed80070ac0 0x7fed801e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 -- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed8015d330 msgr2=0x7fed800b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed8015d330 0x7fed800b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 -- 172.21.15.123:0/1399670017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed801496b0 con 0x7fed8006d580 2024-04-25T19:04:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.666+0000 7fed85800640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b9f0 tx=0x7fed7000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.667+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70003b70 con 0x7fed8006d580 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.667+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed70003d10 con 0x7fed8006d580 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.667+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000cbb0 con 0x7fed8006d580 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.667+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed801e8bb0 con 0x7fed8006d580 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.667+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed801e9170 con 0x7fed8006d580 2024-04-25T19:04:12.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.668+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed80050740 con 0x7fed8006d580 2024-04-25T19:04:12.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.669+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed70003f80 con 0x7fed8006d580 2024-04-25T19:04:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.670+0000 7fed7d600640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 0x7fed600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.670+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6238+0+0 (secure 0 0 0) 0x7fed70095300 con 0x7fed8006d580 2024-04-25T19:04:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.671+0000 7fed84e00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 0x7fed600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:12.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.671+0000 7fed84e00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 0x7fed600769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:12.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.675+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed801496b0 con 0x7fed8006d580 2024-04-25T19:04:12.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:12.876+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed80073ee0 con 0x7fed8006d580 2024-04-25T19:04:13.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.357+0000 7fed7d600640 1 -- 172.21.15.123:0/1399670017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7fed7005f870 con 0x7fed8006d580 2024-04-25T19:04:13.360 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 msgr2=0x7fed600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 0x7fed600769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c008040 comp rx=0 tx=0).stop 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 msgr2=0x7fed80149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b9f0 tx=0x7fed7000beb0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 shutdown_connections 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed600744b0 0x7fed600769c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed8015d330 0x7fed800b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed80070ac0 0x7fed801e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 --2- 172.21.15.123:0/1399670017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8006d580 0x7fed80149170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 >> 172.21.15.123:0/1399670017 conn(0x7fed80072fd0 msgr2=0x7fed8006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:13.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 shutdown_connections 2024-04-25T19:04:13.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.361+0000 7fed86c00640 1 -- 172.21.15.123:0/1399670017 wait complete. 2024-04-25T19:04:13.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T19:04:13.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.663+0000 7f647ce00640 1 Processor -- start 2024-04-25T19:04:13.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.663+0000 7f647ce00640 1 -- start start 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.665+0000 7f647ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64781641a0 0x7f64780b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f647ce00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781645b0 0x7f64780b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f647ce00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6478074520 con 0x7f64780738d0 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f647ce00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6478160620 con 0x7f64781641a0 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f647ce00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f64781613a0 con 0x7f64781645b0 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64781641a0 0x7f64780b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64781641a0 0x7f64780b63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43046/0 (socket says 172.21.15.123:43046) 2024-04-25T19:04:13.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 -- 172.21.15.123:0/2590328793 learned_addr learned my addr 172.21.15.123:0/2590328793 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6476a00640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781645b0 0x7f64780b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 -- 172.21.15.123:0/2590328793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781645b0 msgr2=0x7f64780b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781645b0 0x7f64780b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 -- 172.21.15.123:0/2590328793 >> v1:172.21.15.146:6789/0 conn(0x7f64780738d0 legacy=0x7f6478073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.666+0000 7f6477400640 1 -- 172.21.15.123:0/2590328793 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64781608c0 con 0x7f64781641a0 2024-04-25T19:04:13.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.668+0000 7f6477400640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781641a0 0x7f64780b63b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6468009870 tx=0x7f646802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1a73e6a8f61ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.668+0000 7f6476000640 1 -- 172.21.15.123:0/2590328793 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468037070 con 0x7f64781641a0 2024-04-25T19:04:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.668+0000 7f6476000640 1 -- 172.21.15.123:0/2590328793 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468032440 con 0x7f64781641a0 2024-04-25T19:04:13.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.668+0000 7f6476000640 1 -- 172.21.15.123:0/2590328793 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468032810 con 0x7f64781641a0 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 -- 172.21.15.123:0/2590328793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781641a0 msgr2=0x7f64780b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781641a0 0x7f64780b63b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6468009870 tx=0x7f646802a180 comp rx=0 tx=0).stop 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 -- 172.21.15.123:0/2590328793 shutdown_connections 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781645b0 0x7f64780b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 --2- 172.21.15.123:0/2590328793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781641a0 0x7f64780b63b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 -- 172.21.15.123:0/2590328793 >> 172.21.15.123:0/2590328793 conn(0x7f64780d1dd0 msgr2=0x7f64780d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:13.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 -- 172.21.15.123:0/2590328793 shutdown_connections 2024-04-25T19:04:13.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.669+0000 7f647ce00640 1 -- 172.21.15.123:0/2590328793 wait complete. 2024-04-25T19:04:13.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.672+0000 7f647ce00640 1 Processor -- start 2024-04-25T19:04:13.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.672+0000 7f647ce00640 1 -- start start 2024-04-25T19:04:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 0x7f647814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781641a0 0x7f64780b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64780b4e00 con 0x7f64781645b0 2024-04-25T19:04:13.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6478160e80 con 0x7f64780738d0 2024-04-25T19:04:13.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.674+0000 7f647ce00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f64780b4f80 con 0x7f64781641a0 2024-04-25T19:04:13.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:13.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43060/0 (socket says 172.21.15.123:43060) 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 -- 172.21.15.123:0/292382852 learned_addr learned my addr 172.21.15.123:0/292382852 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477400640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 0x7f647814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 -- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781641a0 msgr2=0x7f64780b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781641a0 0x7f64780b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 -- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 msgr2=0x7f647814d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 0x7f647814d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477e00640 1 -- 172.21.15.123:0/292382852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64780ae310 con 0x7f64781645b0 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.675+0000 7f6477400640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 0x7f647814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f6477e00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f647000b9e0 tx=0x7f647000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470003b70 con 0x7f64781645b0 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6470003d10 con 0x7f64781645b0 2024-04-25T19:04:13.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647000cb00 con 0x7f64781645b0 2024-04-25T19:04:13.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64780ae5f0 con 0x7f64781645b0 2024-04-25T19:04:13.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.676+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64780aebb0 con 0x7f64781645b0 2024-04-25T19:04:13.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.677+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6478050740 con 0x7f64781645b0 2024-04-25T19:04:13.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.679+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f647000cca0 con 0x7f64781645b0 2024-04-25T19:04:13.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.680+0000 7f646f400640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 0x7f64580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:13.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.680+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7f6470094ff0 con 0x7f64781645b0 2024-04-25T19:04:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.680+0000 7f6477400640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 0x7f64580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:13.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.681+0000 7f6477400640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 0x7f64580769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6468034000 tx=0x7f646802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:13.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.685+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f647005f7a0 con 0x7f64781645b0 2024-04-25T19:04:13.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:13.888+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f647806d890 con 0x7f64781645b0 2024-04-25T19:04:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.361+0000 7f646f400640 1 -- 172.21.15.123:0/292382852 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f647005f0a0 con 0x7f64781645b0 2024-04-25T19:04:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' removed 2024-04-25T19:04:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 msgr2=0x7f64580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 0x7f64580769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6468034000 tx=0x7f646802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 msgr2=0x7f64780d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f647000b9e0 tx=0x7f647000bea0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 shutdown_connections 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f64580744b0 0x7f64580769c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64781645b0 0x7f64780d1050 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64781641a0 0x7f64780b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 --2- 172.21.15.123:0/292382852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64780738d0 0x7f647814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.366+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 >> 172.21.15.123:0/292382852 conn(0x7f64780d1dd0 msgr2=0x7f6478162730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.367+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 shutdown_connections 2024-04-25T19:04:14.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.367+0000 7f647ce00640 1 -- 172.21.15.123:0/292382852 wait complete. 2024-04-25T19:04:14.402 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T19:04:14.403 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T19:04:14.403 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T19:04:14.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:04:14.403 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T19:04:14.403 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T19:04:14.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T19:04:14.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:04:14.404 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T19:04:14.404 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T19:04:14.404 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T19:04:14.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:04:14.404 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T19:04:14.404 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T19:04:14.405 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T19:04:14.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:04:14.437 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T19:04:14.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T19:04:14.439 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T19:04:14.439 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T19:04:14.441 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T19:04:14.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T19:04:14.441 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T19:04:14.442 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T19:04:14.485 INFO:tasks.ceph.mds.c:Started 2024-04-25T19:04:14.485 DEBUG:teuthology.parallel:result is None 2024-04-25T19:04:14.487 INFO:tasks.ceph.mds.b:Started 2024-04-25T19:04:14.488 DEBUG:teuthology.parallel:result is None 2024-04-25T19:04:14.491 INFO:tasks.ceph.mds.a:Started 2024-04-25T19:04:14.491 DEBUG:teuthology.parallel:result is None 2024-04-25T19:04:14.494 INFO:tasks.ceph.mds.d:Started 2024-04-25T19:04:14.494 DEBUG:teuthology.parallel:result is None 2024-04-25T19:04:14.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T19:04:14.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T19:04:14.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T19:04:14.718 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-04-25T19:04:14.719 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-04-25T19:04:14.724 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-25T19:04:14.727 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-25T19:04:14.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.765+0000 7f81d4200640 1 Processor -- start 2024-04-25T19:04:14.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.765+0000 7f81d4200640 1 -- start start 2024-04-25T19:04:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.767+0000 7f81d4200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f81cc06d730 0x7f81cc164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.767+0000 7f81d4200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0b59c0 con 0x7f81cc0b5400 2024-04-25T19:04:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.767+0000 7f81d4200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0b5ca0 con 0x7f81cc163e10 2024-04-25T19:04:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.767+0000 7f81d4200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc15ab70 con 0x7f81cc06d730 2024-04-25T19:04:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81d1800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f81cc0b5400 0x7f81cc06fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:38932/0 (socket says 172.21.15.123:38932) 2024-04-25T19:04:14.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81d1800640 1 -- 172.21.15.123:0/2925731048 learned_addr learned my addr 172.21.15.123:0/2925731048 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:14.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81d2200640 1 --2- 172.21.15.123:0/2925731048 >> v2:172.21.15.146:3300/0 conn(0x7f81cc06d730 0x7f81cc164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2241716009 0 0) 0x7f81cc0b59c0 con 0x7f81cc0b5400 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81b4003520 con 0x7f81cc0b5400 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1101166123 0 0) 0x7f81b4003520 con 0x7f81cc0b5400 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 >> v2:172.21.15.146:3300/0 conn(0x7f81cc06d730 msgr2=0x7f81cc164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 --2- 172.21.15.123:0/2925731048 >> v2:172.21.15.146:3300/0 conn(0x7f81cc06d730 0x7f81cc164220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 >> v1:172.21.15.146:6789/0 conn(0x7f81cc163e10 legacy=0x7f81cc0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0b5f80 con 0x7f81cc0b5400 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f81c0001670 con 0x7f81cc0b5400 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.768+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 >> v1:172.21.15.123:6789/0 conn(0x7f81cc0b5400 legacy=0x7f81cc06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 0x7f81b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4004800 0x7f81b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81b4005270 0x7f81b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81b400c160 con 0x7f81b4003a00 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81b400c310 con 0x7f81b4004800 2024-04-25T19:04:14.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81b400c510 con 0x7f81b4005270 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d2200640 1 --2- 172.21.15.123:0/2925731048 >> v2:172.21.15.146:3300/0 conn(0x7f81cc06d730 0x7f81cc164220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 0x7f81b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d0e00640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4004800 0x7f81b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d2200640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81b4005270 0x7f81b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 -- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81b4005270 msgr2=0x7f81b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81b4005270 0x7f81b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 -- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4004800 msgr2=0x7f81b4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4004800 0x7f81b4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d1800640 1 -- 172.21.15.123:0/2925731048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b4003520 con 0x7f81b4003a00 2024-04-25T19:04:14.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.769+0000 7f81d2200640 1 -- 172.21.15.123:0/2925731048 reap_dead start 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.771+0000 7f81d1800640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 0x7f81b4003e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81cc074690 tx=0x7f81b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494fdc4f8ed3af47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.771+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0007c40 con 0x7f81b4003a00 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.771+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c0005af0 con 0x7f81b4003a00 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.771+0000 7f81cbe00640 1 -- 172.21.15.123:0/2925731048 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0007070 con 0x7f81b4003a00 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 -- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 msgr2=0x7f81b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 0x7f81b4003e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81cc074690 tx=0x7f81b4005830 comp rx=0 tx=0).stop 2024-04-25T19:04:14.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 -- 172.21.15.123:0/2925731048 shutdown_connections 2024-04-25T19:04:14.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 --2- 172.21.15.123:0/2925731048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4003a00 0x7f81b4003e70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 -- 172.21.15.123:0/2925731048 >> 172.21.15.123:0/2925731048 conn(0x7f81cc0d1ae0 msgr2=0x7f81cc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 -- 172.21.15.123:0/2925731048 shutdown_connections 2024-04-25T19:04:14.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.772+0000 7f81d4200640 1 -- 172.21.15.123:0/2925731048 wait complete. 2024-04-25T19:04:14.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.774+0000 7f81d4200640 1 Processor -- start 2024-04-25T19:04:14.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.774+0000 7f81d4200640 1 -- start start 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 0x7f81cc0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81cc0b1610 0x7f81cc0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc164220 con 0x7f81cc15b490 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0b6540 con 0x7f81b4003a00 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d4200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc1643a0 con 0x7f81cc0b1610 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43118/0 (socket says 172.21.15.123:43118) 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d2200640 1 -- 172.21.15.123:0/1291878316 learned_addr learned my addr 172.21.15.123:0/1291878316 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d0e00640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81cc0b1610 0x7f81cc0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.776+0000 7f81d1800640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 0x7f81cc0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 -- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81cc0b1610 msgr2=0x7f81cc0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81cc0b1610 0x7f81cc0d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 -- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 msgr2=0x7f81cc0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 0x7f81cc0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 -- 172.21.15.123:0/1291878316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0b1b40 con 0x7f81cc15b490 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d1800640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 0x7f81cc0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d2200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0b5350 tx=0x7f81c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4003990 con 0x7f81cc15b490 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c401b070 con 0x7f81cc15b490 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4003b50 con 0x7f81cc15b490 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc0b1e20 con 0x7f81cc15b490 2024-04-25T19:04:14.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.777+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc0b23e0 con 0x7f81cc15b490 2024-04-25T19:04:14.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.778+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc076a40 con 0x7f81cc15b490 2024-04-25T19:04:14.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.780+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f81c4005d10 con 0x7f81cc15b490 2024-04-25T19:04:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.781+0000 7f81c9600640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 0x7f81ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.781+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f81c4095950 con 0x7f81cc15b490 2024-04-25T19:04:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.781+0000 7f81d1800640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 0x7f81ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.782+0000 7f81d1800640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 0x7f81ac0769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81c0032000 tx=0x7f81c002d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.786+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81c4025020 con 0x7f81cc15b490 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:14.991+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f81cc0d0f80 con 0x7f81cc15b490 2024-04-25T19:04:15.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.409+0000 7f81c9600640 1 -- 172.21.15.123:0/1291878316 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f81c40601b0 con 0x7f81cc15b490 2024-04-25T19:04:15.457 INFO:teuthology.orchestra.run.smithi123.stderr: removed all blocklist entries 2024-04-25T19:04:15.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 msgr2=0x7f81ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 0x7f81ac0769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81c0032000 tx=0x7f81c002d000 comp rx=0 tx=0).stop 2024-04-25T19:04:15.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 msgr2=0x7f81cc15b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0b5350 tx=0x7f81c4009f90 comp rx=0 tx=0).stop 2024-04-25T19:04:15.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 shutdown_connections 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81ac0744b0 0x7f81ac0769c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81cc15b490 0x7f81cc15b8a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81cc0b1610 0x7f81cc0d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 --2- 172.21.15.123:0/1291878316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81b4003a00 0x7f81cc0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.414+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 >> 172.21.15.123:0/1291878316 conn(0x7f81cc0d1ae0 msgr2=0x7f81cc160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.415+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 shutdown_connections 2024-04-25T19:04:15.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.415+0000 7f81d4200640 1 -- 172.21.15.123:0/1291878316 wait complete. 2024-04-25T19:04:15.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T19:04:15.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.733+0000 7f1183600640 1 Processor -- start 2024-04-25T19:04:15.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.733+0000 7f1183600640 1 -- start start 2024-04-25T19:04:15.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.735+0000 7f1183600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f117c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.735+0000 7f1183600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f117c163e10 0x7f117c164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.735+0000 7f1183600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117c0b59c0 con 0x7f117c0b03c0 2024-04-25T19:04:15.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.735+0000 7f1183600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f117c15ab70 con 0x7f117c163e10 2024-04-25T19:04:15.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.735+0000 7f1183600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f117c0b5ca0 con 0x7f117c0b5400 2024-04-25T19:04:15.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f1181600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f117c0b03c0 0x7f117c0b07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43722/0 (socket says 172.21.15.123:43722) 2024-04-25T19:04:15.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f1181600640 1 -- 172.21.15.123:0/1357378986 learned_addr learned my addr 172.21.15.123:0/1357378986 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f1180c00640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f117c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f117be00640 1 --2- 172.21.15.123:0/1357378986 >> v2:172.21.15.146:3300/0 conn(0x7f117c163e10 0x7f117c164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250607279 0 0) 0x7f117c0b59c0 con 0x7f117c0b03c0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.736+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1160003520 con 0x7f117c0b03c0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831422474 0 0) 0x7f1160003520 con 0x7f117c0b03c0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 msgr2=0x7f117c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f117c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 >> v2:172.21.15.146:3300/0 conn(0x7f117c163e10 msgr2=0x7f117c164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 --2- 172.21.15.123:0/1357378986 >> v2:172.21.15.146:3300/0 conn(0x7f117c163e10 0x7f117c164220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c0b5f80 con 0x7f117c0b03c0 2024-04-25T19:04:15.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f11740019f0 con 0x7f117c0b03c0 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 >> v1:172.21.15.123:6789/0 conn(0x7f117c0b03c0 legacy=0x7f117c0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600039b0 0x7f1160003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1160004800 0x7f1160004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f1160005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0b5ca0 con 0x7f11600039b0 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f116000b9c0 con 0x7f1160004800 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f116000bb40 con 0x7f117c0b5400 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.737+0000 7f1180c00640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600039b0 0x7f1160003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1181600640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f1160005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1180c00640 1 -- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 msgr2=0x7f1160005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1180c00640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f1160005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1180c00640 1 -- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1160004800 msgr2=0x7f1160004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1180c00640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1160004800 0x7f1160004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1180c00640 1 -- 172.21.15.123:0/1357378986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1160003520 con 0x7f11600039b0 2024-04-25T19:04:15.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.738+0000 7f1181600640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0b5400 0x7f1160005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.739+0000 7f1180c00640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600039b0 0x7f1160003e20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f117c074690 tx=0x7f1160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd77390966d0757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.739+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c036070 con 0x7f11600039b0 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.739+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c031400 con 0x7f11600039b0 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.739+0000 7f117b400640 1 -- 172.21.15.123:0/1357378986 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c009690 con 0x7f11600039b0 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 -- 172.21.15.123:0/1357378986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600039b0 msgr2=0x7f1160003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 --2- 172.21.15.123:0/1357378986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600039b0 0x7f1160003e20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f117c074690 tx=0x7f1160005830 comp rx=0 tx=0).stop 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1181600640 1 -- 172.21.15.123:0/1357378986 reap_dead start 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 -- 172.21.15.123:0/1357378986 shutdown_connections 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 -- 172.21.15.123:0/1357378986 >> 172.21.15.123:0/1357378986 conn(0x7f117c0d1ae0 msgr2=0x7f117c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 -- 172.21.15.123:0/1357378986 shutdown_connections 2024-04-25T19:04:15.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.740+0000 7f1183600640 1 -- 172.21.15.123:0/1357378986 wait complete. 2024-04-25T19:04:15.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.742+0000 7f1183600640 1 Processor -- start 2024-04-25T19:04:15.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.742+0000 7f1183600640 1 -- start start 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11600039b0 0x7f117c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c15c450 0x7f117c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0b0ee0 con 0x7f117c1494a0 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0b6540 con 0x7f117c15c450 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f1183600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f117c0b1060 con 0x7f11600039b0 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f117be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f117be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55568/0 (socket says 172.21.15.123:55568) 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.744+0000 7f117be00640 1 -- 172.21.15.123:0/3748731113 learned_addr learned my addr 172.21.15.123:0/3748731113 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1181600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c15c450 0x7f117c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 -- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11600039b0 msgr2=0x7f117c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1180c00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11600039b0 0x7f117c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11600039b0 0x7f117c15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 -- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c15c450 msgr2=0x7f117c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c15c450 0x7f117c15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 -- 172.21.15.123:0/3748731113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c15cda0 con 0x7f117c1494a0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f117be00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f116400e890 tx=0x7f116400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f117c1494a0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116400c940 con 0x7f117c1494a0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116400cd10 con 0x7f117c1494a0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c146ba0 con 0x7f117c1494a0 2024-04-25T19:04:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.745+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c147160 con 0x7f117c1494a0 2024-04-25T19:04:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.747+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c15b740 con 0x7f117c1494a0 2024-04-25T19:04:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.748+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f116401a070 con 0x7f117c1494a0 2024-04-25T19:04:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.749+0000 7f1178c00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 0x7f11580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.749+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1164094e80 con 0x7f117c1494a0 2024-04-25T19:04:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.749+0000 7f1180c00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 0x7f11580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.750+0000 7f1180c00640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 0x7f11580769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f116c033000 tx=0x7f116c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:15.753+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f117c15cda0 con 0x7f117c1494a0 2024-04-25T19:04:16.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.007+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f117c070700 con 0x7f117c1494a0 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.008+0000 7f1178c00640 1 -- 172.21.15.123:0/3748731113 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f117c070700 con 0x7f117c1494a0 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_dump": [ 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.a", 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCGqCpmBVn2KRAAWo2mVIL6zYcxXNeUHws5EQ==", 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-25T19:04:16.010 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.b", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCHqCpm4OhZEhAAN00cjmzmkk8s36+QKSljUw==", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.011 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.c", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCGqCpmFt5KOxAAnXURk+vegj7PL5IzJYdqfQ==", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.d", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCHqCpmCfiAJRAAYE9F8O1uISrXo1UJk5eZjw==", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.0", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCMqCpmMne7ARAAD/fsufO2UPnMG5vHj5+/Kg==", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.1", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCPqCpmBzCcEBAAxG14/PkfSEXERSIOZ4SOFg==", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.2", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCSqCpmghBrHxAAzjHRr0EPcDdh0uJ9B0ECEw==", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.014 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.3", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCVqCpmeSyOLxAAvABp5eQNxeOenxSPz4ithA==", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.4", 2024-04-25T19:04:16.015 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCcqCpmWgs9BBAApfP4lek/yUB29DLCEBPJIQ==", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.5", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCfqCpmqvSsIRAA0U6IDjWS2+lzkqcIFEaYIA==", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.6", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCjqCpmUUxEAxAABLXfQutj7ad6CT44UGnW7A==", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.7", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCmqCpmQF5TFhAAxICQfrhe5g/wOs0xChEM6g==", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-25T19:04:16.017 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.0", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCHqCpmMqUvNhAA5Rf0ooG9ZwRKfPr3VFwX6A==", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.1", 2024-04-25T19:04:16.018 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCIqCpmhpLrFhAAOn+ZCacMA0yVBegykPu+OQ==", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.2", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCIqCpmt3C3CBAAF5ln+HFzFpzt2nVjrZDSsQ==", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-25T19:04:16.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.3", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCIqCpmGpSvJBAAiRnXzCJvTv4D6wPmEDOeEw==", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.admin", 2024-04-25T19:04:16.020 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCFqCpmzbhAHRAA26anAytsUh/6ELHsMn43Pg==", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mds", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpm8jeUNhAA3rqmvy0cWDCl3M+LqI4VXA==", 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpmg1SUNhAA7BnU+XBatl249EMAsfFZ2A==", 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-osd", 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpmxHGUNhAAuhO7Sg+vm8QQe1Y7vUYfVw==", 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpmCoqUNhAA56lr4u/C73TymjLKNb1YBQ==", 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpmn6KUNhAAu95Bbo075AyYXsCxF0us+g==", 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.023 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQC1qCpmCr2UNhAAvzjq8y33ZcyBS+CdVhYnOw==", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.x", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCGqCpmS1zAHBAAal0LOYhhk4JQSXYcWs58kw==", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:16.024 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.y", 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCGqCpmUYgMDhAA5gq7FeuAAoyFNZuiFpRDRQ==", 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-25T19:04:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 msgr2=0x7f11580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 0x7f11580769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f116c033000 tx=0x7f116c008030 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 msgr2=0x7f117c142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f116400e890 tx=0x7f116400ed50 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 shutdown_connections 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f11580744b0 0x7f11580769c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c15c450 0x7f117c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117c1494a0 0x7f117c142c40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 --2- 172.21.15.123:0/3748731113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11600039b0 0x7f117c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 >> 172.21.15.123:0/3748731113 conn(0x7f117c0d1ae0 msgr2=0x7f117c1460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.010+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 shutdown_connections 2024-04-25T19:04:16.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.011+0000 7f1183600640 1 -- 172.21.15.123:0/3748731113 wait complete. 2024-04-25T19:04:16.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T19:04:16.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.297+0000 7f1c34000640 1 Processor -- start 2024-04-25T19:04:16.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.298+0000 7f1c34000640 1 -- start start 2024-04-25T19:04:16.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c34000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c2c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c34000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1c2c0cd140 0x7f1c2c0af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c34000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c0cc5f0 con 0x7f1c2c162540 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c34000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c06deb0 con 0x7f1c2c0cd140 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c34000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c06d610 con 0x7f1c2c06daa0 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c30c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1c2c162540 0x7f1c2c0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43724/0 (socket says 172.21.15.123:43724) 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.300+0000 7f1c30c00640 1 -- 172.21.15.123:0/649016324 learned_addr learned my addr 172.21.15.123:0/649016324 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c32000640 1 --2- 172.21.15.123:0/649016324 >> v2:172.21.15.146:3300/0 conn(0x7f1c2c0cd140 0x7f1c2c0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c2c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389467137 0 0) 0x7f1c2c0cc5f0 con 0x7f1c2c162540 2024-04-25T19:04:16.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c10003520 con 0x7f1c2c162540 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3120567316 0 0) 0x7f1c10003520 con 0x7f1c2c162540 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 msgr2=0x7f1c2c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c2c15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 >> v2:172.21.15.146:3300/0 conn(0x7f1c2c0cd140 msgr2=0x7f1c2c0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 --2- 172.21.15.123:0/649016324 >> v2:172.21.15.146:3300/0 conn(0x7f1c2c0cd140 0x7f1c2c0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c06ca00 con 0x7f1c2c162540 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f1c18001670 con 0x7f1c2c162540 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c2c15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 >> v1:172.21.15.123:6789/0 conn(0x7f1c2c162540 legacy=0x7f1c2c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c100039b0 0x7f1c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c10004800 0x7f1c10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c10005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06deb0 con 0x7f1c100039b0 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06d610 con 0x7f1c10004800 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.301+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1000ba70 con 0x7f1c2c06daa0 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c100039b0 0x7f1c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 -- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 msgr2=0x7f1c10005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c06daa0 0x7f1c10005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 -- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c10004800 msgr2=0x7f1c10004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c10004800 0x7f1c10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.302+0000 7f1c31600640 1 -- 172.21.15.123:0/649016324 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c10003520 con 0x7f1c100039b0 2024-04-25T19:04:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.303+0000 7f1c31600640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c100039b0 0x7f1c10003e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0073e0 tx=0x7f1c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1647cc1168031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.303+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c032070 con 0x7f1c100039b0 2024-04-25T19:04:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.303+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c003b90 con 0x7f1c100039b0 2024-04-25T19:04:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.303+0000 7f1c2be00640 1 -- 172.21.15.123:0/649016324 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c009c80 con 0x7f1c100039b0 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 -- 172.21.15.123:0/649016324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c100039b0 msgr2=0x7f1c10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 --2- 172.21.15.123:0/649016324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c100039b0 0x7f1c10003e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0073e0 tx=0x7f1c10005830 comp rx=0 tx=0).stop 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c32000640 1 -- 172.21.15.123:0/649016324 reap_dead start 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 -- 172.21.15.123:0/649016324 shutdown_connections 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 -- 172.21.15.123:0/649016324 >> 172.21.15.123:0/649016324 conn(0x7f1c2c0ce390 msgr2=0x7f1c2c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 -- 172.21.15.123:0/649016324 shutdown_connections 2024-04-25T19:04:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.304+0000 7f1c34000640 1 -- 172.21.15.123:0/649016324 wait complete. 2024-04-25T19:04:16.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.306+0000 7f1c34000640 1 Processor -- start 2024-04-25T19:04:16.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.306+0000 7f1c34000640 1 -- start start 2024-04-25T19:04:16.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c100039b0 0x7f1c2c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c159310 0x7f1c2c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06d2a0 con 0x7f1c2c154ab0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06cfc0 con 0x7f1c100039b0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c34000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c162950 con 0x7f1c2c159310 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c32000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c32000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55574/0 (socket says 172.21.15.123:55574) 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c32000640 1 -- 172.21.15.123:0/106950821 learned_addr learned my addr 172.21.15.123:0/106950821 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c32000640 1 -- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c159310 msgr2=0x7f1c2c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.308+0000 7f1c30c00640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c159310 0x7f1c2c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c32000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c159310 0x7f1c2c159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c31600640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c100039b0 0x7f1c2c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c32000640 1 -- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c100039b0 msgr2=0x7f1c2c146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c32000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c100039b0 0x7f1c2c146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c32000640 1 -- 172.21.15.123:0/106950821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c159c60 con 0x7f1c2c154ab0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c32000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ae870 tx=0x7f1c2400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400c680 con 0x7f1c2c154ab0 2024-04-25T19:04:16.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2401a070 con 0x7f1c2c154ab0 2024-04-25T19:04:16.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400ca60 con 0x7f1c2c154ab0 2024-04-25T19:04:16.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c1e8520 con 0x7f1c2c154ab0 2024-04-25T19:04:16.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.309+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1e8ae0 con 0x7f1c2c154ab0 2024-04-25T19:04:16.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.311+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c076a40 con 0x7f1c2c154ab0 2024-04-25T19:04:16.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.312+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c2400cc00 con 0x7f1c2c154ab0 2024-04-25T19:04:16.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.313+0000 7f1c29600640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 0x7f1c0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.313+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1c24094690 con 0x7f1c2c154ab0 2024-04-25T19:04:16.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.313+0000 7f1c31600640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 0x7f1c0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.314+0000 7f1c31600640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 0x7f1c0c0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c034000 tx=0x7f1c1c00b070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.317+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c2c159c60 con 0x7f1c2c154ab0 2024-04-25T19:04:16.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.585+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f1c2c069c60 con 0x7f1c2c154ab0 2024-04-25T19:04:16.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.590+0000 7f1c29600640 1 -- 172.21.15.123:0/106950821 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f1c2405efd0 con 0x7f1c2c154ab0 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 msgr2=0x7f1c0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 0x7f1c0c0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c034000 tx=0x7f1c1c00b070 comp rx=0 tx=0).stop 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 msgr2=0x7f1c2c154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ae870 tx=0x7f1c2400ec00 comp rx=0 tx=0).stop 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 shutdown_connections 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1c0c0744b0 0x7f1c0c0769c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c2c154ab0 0x7f1c2c154ec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c2c159310 0x7f1c2c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 --2- 172.21.15.123:0/106950821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c100039b0 0x7f1c2c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 >> 172.21.15.123:0/106950821 conn(0x7f1c2c0ce390 msgr2=0x7f1c2c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 shutdown_connections 2024-04-25T19:04:16.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.593+0000 7f1c34000640 1 -- 172.21.15.123:0/106950821 wait complete. 2024-04-25T19:04:16.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T19:04:16.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.875+0000 7f09f7200640 1 Processor -- start 2024-04-25T19:04:16.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.875+0000 7f09f7200640 1 -- start start 2024-04-25T19:04:16.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.877+0000 7f09f7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.877+0000 7f09f7200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09f806dc90 0x7f09f80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.877+0000 7f09f7200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f815e2e0 con 0x7f09f8143040 2024-04-25T19:04:16.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f7200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f09f80d1200 con 0x7f09f806dc90 2024-04-25T19:04:16.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f09f80d05f0 con 0x7f09f80cfff0 2024-04-25T19:04:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f5e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f6800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09f806dc90 0x7f09f80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f5e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:41648/0 (socket says 172.21.15.123:41648) 2024-04-25T19:04:16.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.878+0000 7f09f5e00640 1 -- 172.21.15.123:0/3920544670 learned_addr learned my addr 172.21.15.123:0/3920544670 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:16.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.879+0000 7f09f5e00640 1 -- 172.21.15.123:0/3920544670 >> v2:172.21.15.146:3300/0 conn(0x7f09f806dc90 msgr2=0x7f09f80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.879+0000 7f09f5e00640 1 --2- 172.21.15.123:0/3920544670 >> v2:172.21.15.146:3300/0 conn(0x7f09f806dc90 0x7f09f80cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.879+0000 7f09f5e00640 1 -- 172.21.15.123:0/3920544670 >> v1:172.21.15.146:6789/0 conn(0x7f09f8143040 legacy=0x7f09f806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.879+0000 7f09f5e00640 1 -- 172.21.15.123:0/3920544670 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f80d08d0 con 0x7f09f80cfff0 2024-04-25T19:04:16.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.881+0000 7f09f5e00640 1 --2- 172.21.15.123:0/3920544670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009870 tx=0x7f09e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2bd77f47f297e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.881+0000 7f09f4a00640 1 -- 172.21.15.123:0/3920544670 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8037070 con 0x7f09f80cfff0 2024-04-25T19:04:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.881+0000 7f09f4a00640 1 -- 172.21.15.123:0/3920544670 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e8032440 con 0x7f09f80cfff0 2024-04-25T19:04:16.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.881+0000 7f09f4a00640 1 -- 172.21.15.123:0/3920544670 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8032810 con 0x7f09f80cfff0 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 -- 172.21.15.123:0/3920544670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 msgr2=0x7f09f815dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 --2- 172.21.15.123:0/3920544670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009870 tx=0x7f09e802a270 comp rx=0 tx=0).stop 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 -- 172.21.15.123:0/3920544670 shutdown_connections 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 --2- 172.21.15.123:0/3920544670 >> v2:172.21.15.146:3300/0 conn(0x7f09f806dc90 0x7f09f80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 --2- 172.21.15.123:0/3920544670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80cfff0 0x7f09f815dda0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 -- 172.21.15.123:0/3920544670 >> 172.21.15.123:0/3920544670 conn(0x7f09f8073090 msgr2=0x7f09f80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 -- 172.21.15.123:0/3920544670 shutdown_connections 2024-04-25T19:04:16.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.882+0000 7f09f7200640 1 -- 172.21.15.123:0/3920544670 wait complete. 2024-04-25T19:04:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.885+0000 7f09f7200640 1 Processor -- start 2024-04-25T19:04:16.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.885+0000 7f09f7200640 1 -- start start 2024-04-25T19:04:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f8143040 0x7f09f81f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 0x7f09f80b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8143450 con 0x7f09f813e920 2024-04-25T19:04:16.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80d0e90 con 0x7f09f8143040 2024-04-25T19:04:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.887+0000 7f09f7200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f09f813a180 con 0x7f09f80b44a0 2024-04-25T19:04:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55582/0 (socket says 172.21.15.123:55582) 2024-04-25T19:04:16.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 -- 172.21.15.123:0/271581575 learned_addr learned my addr 172.21.15.123:0/271581575 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5e00640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f8143040 0x7f09f81f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f6800640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 0x7f09f80b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 -- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 msgr2=0x7f09f80b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 0x7f09f80b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 -- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f8143040 msgr2=0x7f09f81f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f8143040 0x7f09f81f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f5400640 1 -- 172.21.15.123:0/271581575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f813d880 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.888+0000 7f09f6800640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 0x7f09f80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09f5400640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c3f0 tx=0x7f09e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00052d0 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0005490 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0007260 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f813db60 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.889+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f813e120 con 0x7f09f813e920 2024-04-25T19:04:16.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.890+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f8050740 con 0x7f09f813e920 2024-04-25T19:04:16.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.891+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09e0007400 con 0x7f09f813e920 2024-04-25T19:04:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.892+0000 7f09ee000640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 0x7f09d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.892+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f09e00945c0 con 0x7f09f813e920 2024-04-25T19:04:16.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.893+0000 7f09f5e00640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 0x7f09d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.893+0000 7f09f5e00640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 0x7f09d40769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09e8034000 tx=0x7f09e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:16.897+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09f815e2e0 con 0x7f09f813e920 2024-04-25T19:04:17.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.175+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f09f80742b0 con 0x7f09f813e920 2024-04-25T19:04:17.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.181+0000 7f09ee000640 1 -- 172.21.15.123:0/271581575 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f09e005ee70 con 0x7f09f813e920 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 msgr2=0x7f09d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 0x7f09d40769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09e8034000 tx=0x7f09e802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 msgr2=0x7f09f81395e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c3f0 tx=0x7f09e000c8b0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 shutdown_connections 2024-04-25T19:04:17.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f09d40744b0 0x7f09d40769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b44a0 0x7f09f80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f813e920 0x7f09f81395e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 --2- 172.21.15.123:0/271581575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f8143040 0x7f09f81f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 >> 172.21.15.123:0/271581575 conn(0x7f09f8073090 msgr2=0x7f09f80cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 shutdown_connections 2024-04-25T19:04:17.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.184+0000 7f09f7200640 1 -- 172.21.15.123:0/271581575 wait complete. 2024-04-25T19:04:17.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T19:04:17.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.469+0000 7f5589a00640 1 Processor -- start 2024-04-25T19:04:17.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.470+0000 7f5589a00640 1 -- start start 2024-04-25T19:04:17.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.471+0000 7f5589a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f558415df10 0x7f55840702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.471+0000 7f5589a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f5584162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.471+0000 7f5589a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55840d1430 con 0x7f558406d580 2024-04-25T19:04:17.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.471+0000 7f5589a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5584070f40 con 0x7f558415df10 2024-04-25T19:04:17.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.471+0000 7f5589a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5584161e90 con 0x7f55840b0700 2024-04-25T19:04:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.472+0000 7f5589000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f558406d580 0x7f558406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43726/0 (socket says 172.21.15.123:43726) 2024-04-25T19:04:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.472+0000 7f5589000640 1 -- 172.21.15.123:0/485677193 learned_addr learned my addr 172.21.15.123:0/485677193 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.472+0000 7f5583e00640 1 --2- 172.21.15.123:0/485677193 >> v2:172.21.15.146:3300/0 conn(0x7f558415df10 0x7f55840702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.472+0000 7f557b400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f5584162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100086745 0 0) 0x7f55840d1430 con 0x7f558406d580 2024-04-25T19:04:17.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5568003520 con 0x7f558406d580 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 110245369 0 0) 0x7f5568003520 con 0x7f558406d580 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 msgr2=0x7f5584162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f5584162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 >> v2:172.21.15.146:3300/0 conn(0x7f558415df10 msgr2=0x7f55840702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 --2- 172.21.15.123:0/485677193 >> v2:172.21.15.146:3300/0 conn(0x7f558415df10 0x7f55840702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584070980 con 0x7f558406d580 2024-04-25T19:04:17.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f557c0019f0 con 0x7f558406d580 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 >> v1:172.21.15.123:6789/0 conn(0x7f558406d580 legacy=0x7f558406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f55680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5568003f30 0x7f5568004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f5568005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5584161e90 con 0x7f5568004d40 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f556800fdd0 con 0x7f5568003f30 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.473+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f556800ffa0 con 0x7f55840b0700 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f5568005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5583e00640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f55680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f557b400640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5568003f30 0x7f5568004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 -- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 msgr2=0x7f55680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55840b0700 0x7f55680039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 -- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5568003f30 msgr2=0x7f5568004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5568003f30 0x7f5568004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.474+0000 7f5589000640 1 -- 172.21.15.123:0/485677193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5568003520 con 0x7f5568004d40 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.475+0000 7f5589000640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f5568005150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557c00a870 tx=0x7f5568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b12fb47ac3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.475+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00a4c0 con 0x7f5568004d40 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.475+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c02f070 con 0x7f5568004d40 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.475+0000 7f5583400640 1 -- 172.21.15.123:0/485677193 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c02bd20 con 0x7f5568004d40 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 -- 172.21.15.123:0/485677193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 msgr2=0x7f5568005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 --2- 172.21.15.123:0/485677193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f5568005150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557c00a870 tx=0x7f5568005830 comp rx=0 tx=0).stop 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589000640 1 -- 172.21.15.123:0/485677193 reap_dead start 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 -- 172.21.15.123:0/485677193 shutdown_connections 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 -- 172.21.15.123:0/485677193 >> 172.21.15.123:0/485677193 conn(0x7f5584073090 msgr2=0x7f55840734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 -- 172.21.15.123:0/485677193 shutdown_connections 2024-04-25T19:04:17.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.476+0000 7f5589a00640 1 -- 172.21.15.123:0/485677193 wait complete. 2024-04-25T19:04:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.479+0000 7f5589a00640 1 Processor -- start 2024-04-25T19:04:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.479+0000 7f5589a00640 1 -- start start 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55841fe7c0 0x7f55841f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55841583f0 0x7f5584158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5584162320 con 0x7f5568004d40 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5584161b20 con 0x7f55841fe7c0 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5589a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f55841624a0 con 0x7f55841583f0 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55610/0 (socket says 172.21.15.123:55610) 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 -- 172.21.15.123:0/164658639 learned_addr learned my addr 172.21.15.123:0/164658639 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 -- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55841583f0 msgr2=0x7f5584158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55841583f0 0x7f5584158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 -- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55841fe7c0 msgr2=0x7f55841f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55841fe7c0 0x7f55841f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.482+0000 7f5583e00640 1 -- 172.21.15.123:0/164658639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584158d40 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5583e00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574010850 tx=0x7f5574010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557400f040 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557400eb00 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55841605e0 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557401c400 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.483+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5584160ba0 con 0x7f5568004d40 2024-04-25T19:04:17.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.484+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5584050740 con 0x7f5568004d40 2024-04-25T19:04:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.485+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f557401c5a0 con 0x7f5568004d40 2024-04-25T19:04:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.486+0000 7f5580c00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 0x7f55600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.486+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f55740956d0 con 0x7f5568004d40 2024-04-25T19:04:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.487+0000 7f557b400640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 0x7f55600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.487+0000 7f557b400640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 0x7f55600769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5584159400 tx=0x7f556c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:17.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.491+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5584158d40 con 0x7f5568004d40 2024-04-25T19:04:17.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.765+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f5584074190 con 0x7f5568004d40 2024-04-25T19:04:17.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.771+0000 7f5580c00640 1 -- 172.21.15.123:0/164658639 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f5574018030 con 0x7f5568004d40 2024-04-25T19:04:17.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 msgr2=0x7f55600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 0x7f55600769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5584159400 tx=0x7f556c008040 comp rx=0 tx=0).stop 2024-04-25T19:04:17.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 msgr2=0x7f558415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574010850 tx=0x7f5574010d10 comp rx=0 tx=0).stop 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 shutdown_connections 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f55600744b0 0x7f55600769c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55841583f0 0x7f5584158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55841fe7c0 0x7f55841f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 --2- 172.21.15.123:0/164658639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568004d40 0x7f558415fd90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.773+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 >> 172.21.15.123:0/164658639 conn(0x7f5584073090 msgr2=0x7f5584156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.774+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 shutdown_connections 2024-04-25T19:04:17.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:17.774+0000 7f5589a00640 1 -- 172.21.15.123:0/164658639 wait complete. 2024-04-25T19:04:17.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T19:04:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.059+0000 7f4f42c00640 1 Processor -- start 2024-04-25T19:04:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.060+0000 7f4f42c00640 1 -- start start 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d0e50 0x7f4f3c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f3c1608e0 0x7f4f3c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3c06ded0 con 0x7f4f3c06dac0 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3c070840 con 0x7f4f3c1608e0 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.062+0000 7f4f42c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b67a0 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55616/0 (socket says 172.21.15.123:55616) 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 -- 172.21.15.123:0/2508638726 learned_addr learned my addr 172.21.15.123:0/2508638726 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f41800640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d0e50 0x7f4f3c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 -- 172.21.15.123:0/2508638726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d0e50 msgr2=0x7f4f3c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d0e50 0x7f4f3c15d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 -- 172.21.15.123:0/2508638726 >> v2:172.21.15.146:3300/0 conn(0x7f4f3c1608e0 msgr2=0x7f4f3c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f40e00640 1 --2- 172.21.15.123:0/2508638726 >> v2:172.21.15.146:3300/0 conn(0x7f4f3c1608e0 0x7f4f3c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 --2- 172.21.15.123:0/2508638726 >> v2:172.21.15.146:3300/0 conn(0x7f4f3c1608e0 0x7f4f3c06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.063+0000 7f4f42200640 1 -- 172.21.15.123:0/2508638726 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c0b64c0 con 0x7f4f3c06dac0 2024-04-25T19:04:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.064+0000 7f4f42200640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0d2700 tx=0x7f4f3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce39cc19306ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.064+0000 7f4f3be00640 1 -- 172.21.15.123:0/2508638726 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34036070 con 0x7f4f3c06dac0 2024-04-25T19:04:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f3be00640 1 -- 172.21.15.123:0/2508638726 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f34031600 con 0x7f4f3c06dac0 2024-04-25T19:04:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f3be00640 1 -- 172.21.15.123:0/2508638726 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340319d0 con 0x7f4f3c06dac0 2024-04-25T19:04:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 -- 172.21.15.123:0/2508638726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c06dac0 msgr2=0x7f4f3c0b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0d2700 tx=0x7f4f3402a6c0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 -- 172.21.15.123:0/2508638726 shutdown_connections 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b44f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2508638726 >> v2:172.21.15.146:3300/0 conn(0x7f4f3c1608e0 0x7f4f3c06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2508638726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d0e50 0x7f4f3c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 -- 172.21.15.123:0/2508638726 >> 172.21.15.123:0/2508638726 conn(0x7f4f3c073090 msgr2=0x7f4f3c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 -- 172.21.15.123:0/2508638726 shutdown_connections 2024-04-25T19:04:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.065+0000 7f4f42c00640 1 -- 172.21.15.123:0/2508638726 wait complete. 2024-04-25T19:04:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.068+0000 7f4f42c00640 1 Processor -- start 2024-04-25T19:04:18.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.068+0000 7f4f42c00640 1 -- start start 2024-04-25T19:04:18.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c1608e0 0x7f4f3c1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c15d000 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c15cd20 con 0x7f4f3c06dac0 2024-04-25T19:04:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.069+0000 7f4f42c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c160cf0 con 0x7f4f3c1608e0 2024-04-25T19:04:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55624/0 (socket says 172.21.15.123:55624) 2024-04-25T19:04:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 -- 172.21.15.123:0/2832438676 learned_addr learned my addr 172.21.15.123:0/2832438676 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f41800640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f42200640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c1608e0 0x7f4f3c1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 -- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c1608e0 msgr2=0x7f4f3c1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c1608e0 0x7f4f3c1442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 -- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 msgr2=0x7f4f3c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f40e00640 1 -- 172.21.15.123:0/2832438676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c14a0d0 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.070+0000 7f4f41800640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f40e00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c820 tx=0x7f4f2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28015070 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f28004020 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c14a3b0 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280043f0 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.071+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c14a970 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.072+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c050740 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.073+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f28007830 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.074+0000 7f4f39600640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.075+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f4f28094e10 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.075+0000 7f4f41800640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.075+0000 7f4f41800640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000b820 tx=0x7f4f30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.079+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f28098030 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.351+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f4f3c0d1370 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.356+0000 7f4f39600640 1 -- 172.21.15.123:0/2832438676 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f4f2805f750 con 0x7f4f3c0d0e50 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 msgr2=0x7f4f1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000b820 tx=0x7f4f30008040 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 msgr2=0x7f4f3c14b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c820 tx=0x7f4f2800cce0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 shutdown_connections 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c1608e0 0x7f4f3c1442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f3c0d0e50 0x7f4f3c14b8a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 --2- 172.21.15.123:0/2832438676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c06dac0 0x7f4f3c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 >> 172.21.15.123:0/2832438676 conn(0x7f4f3c073090 msgr2=0x7f4f3c06f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:18.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 shutdown_connections 2024-04-25T19:04:18.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.359+0000 7f4f42c00640 1 -- 172.21.15.123:0/2832438676 wait complete. 2024-04-25T19:04:18.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T19:04:18.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.658+0000 7f989fe00640 1 Processor -- start 2024-04-25T19:04:18.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.659+0000 7f989fe00640 1 -- start start 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98a0071d10 0x7f98a0160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00d26c0 con 0x7f98a00b6680 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989fe00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00d2140 con 0x7f98a00cf2d0 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f98a00d23e0 con 0x7f98a0071d10 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f9897e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98a0071d10 0x7f98a0160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f98a00b6680 0x7f98a0071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43728/0 (socket says 172.21.15.123:43728) 2024-04-25T19:04:18.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.661+0000 7f989ea00640 1 -- 172.21.15.123:0/3000264212 learned_addr learned my addr 172.21.15.123:0/3000264212 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:18.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.662+0000 7f9897e00640 1 -- 172.21.15.123:0/3000264212 >> v1:172.21.15.146:6789/0 conn(0x7f98a00cf2d0 legacy=0x7f98a00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:18.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.662+0000 7f9897e00640 1 -- 172.21.15.123:0/3000264212 >> v1:172.21.15.123:6789/0 conn(0x7f98a00b6680 legacy=0x7f98a0071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.662+0000 7f9897e00640 1 -- 172.21.15.123:0/3000264212 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00cf930 con 0x7f98a0071d10 2024-04-25T19:04:18.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.662+0000 7f989e000640 1 -- 172.21.15.123:0/3000264212 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132833475 0 0) 0x7f98a00d26c0 con 0x7f98a00b6680 2024-04-25T19:04:18.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.663+0000 7f9897e00640 1 --2- 172.21.15.123:0/3000264212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a0071d10 0x7f98a0160180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f98880097a0 tx=0x7f988802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2714c69b6da3a6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.663+0000 7f989e000640 1 -- 172.21.15.123:0/3000264212 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888037070 con 0x7f98a0071d10 2024-04-25T19:04:18.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.663+0000 7f989e000640 1 -- 172.21.15.123:0/3000264212 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888032440 con 0x7f98a0071d10 2024-04-25T19:04:18.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.663+0000 7f989e000640 1 -- 172.21.15.123:0/3000264212 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888032810 con 0x7f98a0071d10 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 -- 172.21.15.123:0/3000264212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a0071d10 msgr2=0x7f98a0160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 --2- 172.21.15.123:0/3000264212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a0071d10 0x7f98a0160180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f98880097a0 tx=0x7f988802a1e0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 -- 172.21.15.123:0/3000264212 shutdown_connections 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 --2- 172.21.15.123:0/3000264212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a0071d10 0x7f98a0160180 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 -- 172.21.15.123:0/3000264212 >> 172.21.15.123:0/3000264212 conn(0x7f98a00d02b0 msgr2=0x7f98a00d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 -- 172.21.15.123:0/3000264212 shutdown_connections 2024-04-25T19:04:18.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.664+0000 7f989fe00640 1 -- 172.21.15.123:0/3000264212 wait complete. 2024-04-25T19:04:18.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.666+0000 7f989fe00640 1 Processor -- start 2024-04-25T19:04:18.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.667+0000 7f989fe00640 1 -- start start 2024-04-25T19:04:18.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a00b6680 0x7f98a006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98a014efb0 0x7f98a0070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00d3330 con 0x7f98a00cf2d0 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00d31b0 con 0x7f98a00b6680 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f989fe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00ce590 con 0x7f98a014efb0 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f9897e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f9897e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55652/0 (socket says 172.21.15.123:55652) 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.668+0000 7f9897e00640 1 -- 172.21.15.123:0/1049694999 learned_addr learned my addr 172.21.15.123:0/1049694999 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f989ea00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a00b6680 0x7f98a006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f989f400640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98a014efb0 0x7f98a0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 -- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98a014efb0 msgr2=0x7f98a0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98a014efb0 0x7f98a0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 -- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a00b6680 msgr2=0x7f98a006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a00b6680 0x7f98a006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 -- 172.21.15.123:0/1049694999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a006e560 con 0x7f98a00cf2d0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.669+0000 7f9897e00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f98880097a0 tx=0x7f98880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.670+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888037040 con 0x7f98a00cf2d0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.670+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888004250 con 0x7f98a00cf2d0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.670+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888032e20 con 0x7f98a00cf2d0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.670+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a006e840 con 0x7f98a00cf2d0 2024-04-25T19:04:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.670+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a006ee00 con 0x7f98a00cf2d0 2024-04-25T19:04:18.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.671+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a006e2d0 con 0x7f98a00cf2d0 2024-04-25T19:04:18.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.672+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9888030770 con 0x7f98a00cf2d0 2024-04-25T19:04:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.673+0000 7f9897400640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 0x7f987c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.673+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f98880b60d0 con 0x7f98a00cf2d0 2024-04-25T19:04:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.674+0000 7f989ea00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 0x7f987c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.674+0000 7f989ea00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 0x7f987c0769f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9890000f60 tx=0x7f9890007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:18.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.680+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9888080a00 con 0x7f98a00cf2d0 2024-04-25T19:04:18.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.958+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f98a0050740 con 0x7f98a00cf2d0 2024-04-25T19:04:18.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.964+0000 7f9897400640 1 -- 172.21.15.123:0/1049694999 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f9888080300 con 0x7f98a00cf2d0 2024-04-25T19:04:18.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 msgr2=0x7f987c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 0x7f987c0769f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9890000f60 tx=0x7f9890007c00 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 msgr2=0x7f98a014ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f98880097a0 tx=0x7f98880046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 shutdown_connections 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f987c0744e0 0x7f987c0769f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98a014efb0 0x7f98a0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a00cf2d0 0x7f98a014ea70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 --2- 172.21.15.123:0/1049694999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a00b6680 0x7f98a006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.967+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 >> 172.21.15.123:0/1049694999 conn(0x7f98a00d02b0 msgr2=0x7f98a00b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.968+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 shutdown_connections 2024-04-25T19:04:18.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:18.968+0000 7f989fe00640 1 -- 172.21.15.123:0/1049694999 wait complete. 2024-04-25T19:04:18.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T19:04:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.250+0000 7f1d6c400640 1 Processor -- start 2024-04-25T19:04:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.250+0000 7f1d6c400640 1 -- start start 2024-04-25T19:04:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.252+0000 7f1d6c400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d640cd140 0x7f1d6415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.252+0000 7f1d6c400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64162540 0x7f1d640af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.252+0000 7f1d6c400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640cc5f0 con 0x7f1d6406daa0 2024-04-25T19:04:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.252+0000 7f1d6c400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6406deb0 con 0x7f1d640cd140 2024-04-25T19:04:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.252+0000 7f1d6c400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6406d610 con 0x7f1d64162540 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1d6406daa0 0x7f1d640cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43730/0 (socket says 172.21.15.123:43730) 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d640cd140 0x7f1d6415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69a00640 1 -- 172.21.15.123:0/2937119710 learned_addr learned my addr 172.21.15.123:0/2937119710 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d6a400640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64162540 0x7f1d640af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69000640 1 -- 172.21.15.123:0/2937119710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64162540 msgr2=0x7f1d640af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69000640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64162540 0x7f1d640af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69000640 1 -- 172.21.15.123:0/2937119710 >> v1:172.21.15.123:6789/0 conn(0x7f1d6406daa0 legacy=0x7f1d640cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d63e00640 1 -- 172.21.15.123:0/2937119710 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872810045 0 0) 0x7f1d640cc5f0 con 0x7f1d6406daa0 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.253+0000 7f1d69000640 1 -- 172.21.15.123:0/2937119710 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6406ca00 con 0x7f1d640cd140 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.254+0000 7f1d69000640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d640cd140 0x7f1d6415dfc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d580097a0 tx=0x7f1d5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306571d1e69211a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d63e00640 1 -- 172.21.15.123:0/2937119710 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58037070 con 0x7f1d640cd140 2024-04-25T19:04:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d63e00640 1 -- 172.21.15.123:0/2937119710 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58032440 con 0x7f1d640cd140 2024-04-25T19:04:19.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d63e00640 1 -- 172.21.15.123:0/2937119710 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58032810 con 0x7f1d640cd140 2024-04-25T19:04:19.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 -- 172.21.15.123:0/2937119710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d640cd140 msgr2=0x7f1d6415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d640cd140 0x7f1d6415dfc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d580097a0 tx=0x7f1d5802a180 comp rx=0 tx=0).stop 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 -- 172.21.15.123:0/2937119710 shutdown_connections 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64162540 0x7f1d640af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 --2- 172.21.15.123:0/2937119710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d640cd140 0x7f1d6415dfc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.255+0000 7f1d6c400640 1 -- 172.21.15.123:0/2937119710 >> 172.21.15.123:0/2937119710 conn(0x7f1d640ce390 msgr2=0x7f1d640ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.256+0000 7f1d6c400640 1 -- 172.21.15.123:0/2937119710 shutdown_connections 2024-04-25T19:04:19.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.256+0000 7f1d6c400640 1 -- 172.21.15.123:0/2937119710 wait complete. 2024-04-25T19:04:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.258+0000 7f1d6c400640 1 Processor -- start 2024-04-25T19:04:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.258+0000 7f1d6c400640 1 -- start start 2024-04-25T19:04:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d6406daa0 0x7f1d6414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64160530 0x7f1d64154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6406d2a0 con 0x7f1d64162540 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6406cfc0 con 0x7f1d6406daa0 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.259+0000 7f1d6c400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1d64162950 con 0x7f1d64160530 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55658/0 (socket says 172.21.15.123:55658) 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 -- 172.21.15.123:0/571876239 learned_addr learned my addr 172.21.15.123:0/571876239 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 -- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64160530 msgr2=0x7f1d64154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64160530 0x7f1d64154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 -- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d6406daa0 msgr2=0x7f1d6414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d6406daa0 0x7f1d6414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 -- 172.21.15.123:0/571876239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6414c8c0 con 0x7f1d64162540 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.260+0000 7f1d69000640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098c0 tx=0x7f1d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.261+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58037040 con 0x7f1d64162540 2024-04-25T19:04:19.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.261+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58042070 con 0x7f1d64162540 2024-04-25T19:04:19.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.261+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58030a70 con 0x7f1d64162540 2024-04-25T19:04:19.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.261+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d64160c40 con 0x7f1d64162540 2024-04-25T19:04:19.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.261+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d640b5360 con 0x7f1d64162540 2024-04-25T19:04:19.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.262+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d641609b0 con 0x7f1d64162540 2024-04-25T19:04:19.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.263+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1d58030c10 con 0x7f1d64162540 2024-04-25T19:04:19.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.265+0000 7f1d61600640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 0x7f1d440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.265+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1d580b59a0 con 0x7f1d64162540 2024-04-25T19:04:19.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.265+0000 7f1d69a00640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 0x7f1d440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.266+0000 7f1d69a00640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 0x7f1d440769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d50007fd0 tx=0x7f1d50007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.270+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d58080370 con 0x7f1d64162540 2024-04-25T19:04:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.527+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1d64162360 con 0x7f1d64162540 2024-04-25T19:04:19.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.533+0000 7f1d61600640 1 -- 172.21.15.123:0/571876239 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f1d64162360 con 0x7f1d64162540 2024-04-25T19:04:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 msgr2=0x7f1d440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 0x7f1d440769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d50007fd0 tx=0x7f1d50007960 comp rx=0 tx=0).stop 2024-04-25T19:04:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 msgr2=0x7f1d6415fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098c0 tx=0x7f1d580046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 shutdown_connections 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f1d440744b0 0x7f1d440769c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d64160530 0x7f1d64154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64162540 0x7f1d6415fff0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 --2- 172.21.15.123:0/571876239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d6406daa0 0x7f1d6414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.536+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 >> 172.21.15.123:0/571876239 conn(0x7f1d640ce390 msgr2=0x7f1d6415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.537+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 shutdown_connections 2024-04-25T19:04:19.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.537+0000 7f1d6c400640 1 -- 172.21.15.123:0/571876239 wait complete. 2024-04-25T19:04:19.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:19.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.814+0000 7f8c7f000640 1 Processor -- start 2024-04-25T19:04:19.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.814+0000 7f8c7f000640 1 -- start start 2024-04-25T19:04:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.816+0000 7f8c7f000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c7806d580 0x7f8c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.816+0000 7f8c7f000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7815e2f0 0x7f8c7815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.816+0000 7f8c7f000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7806ded0 con 0x7f8c7806fe20 2024-04-25T19:04:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.816+0000 7f8c7f000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7815f070 con 0x7f8c7806d580 2024-04-25T19:04:19.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.816+0000 7f8c7f000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7806f340 con 0x7f8c7815e2f0 2024-04-25T19:04:19.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.817+0000 7f8c7d200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c7806d580 0x7f8c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.817+0000 7f8c7d200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c7806d580 0x7f8c7806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:41772/0 (socket says 172.21.15.123:41772) 2024-04-25T19:04:19.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.817+0000 7f8c7d200640 1 -- 172.21.15.123:0/3221544493 learned_addr learned my addr 172.21.15.123:0/3221544493 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:19.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.817+0000 7f8c7e600640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7815e2f0 0x7f8c7815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.818+0000 7f8c7d200640 1 -- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7815e2f0 msgr2=0x7f8c7815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.818+0000 7f8c7d200640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7815e2f0 0x7f8c7815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.818+0000 7f8c7d200640 1 -- 172.21.15.123:0/3221544493 >> v1:172.21.15.146:6789/0 conn(0x7f8c7806fe20 legacy=0x7f8c7815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.818+0000 7f8c7d200640 1 -- 172.21.15.123:0/3221544493 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7806f620 con 0x7f8c7806d580 2024-04-25T19:04:19.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.819+0000 7f8c7d200640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7806d580 0x7f8c7806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009870 tx=0x7f8c6402a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fffa822532491d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.820+0000 7f8c77e00640 1 -- 172.21.15.123:0/3221544493 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64037070 con 0x7f8c7806d580 2024-04-25T19:04:19.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.820+0000 7f8c77e00640 1 -- 172.21.15.123:0/3221544493 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c64032440 con 0x7f8c7806d580 2024-04-25T19:04:19.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.820+0000 7f8c77e00640 1 -- 172.21.15.123:0/3221544493 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64032810 con 0x7f8c7806d580 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 -- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7806d580 msgr2=0x7f8c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7806d580 0x7f8c7806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009870 tx=0x7f8c6402a1d0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 -- 172.21.15.123:0/3221544493 shutdown_connections 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7815e2f0 0x7f8c7815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 --2- 172.21.15.123:0/3221544493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7806d580 0x7f8c7806d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 -- 172.21.15.123:0/3221544493 >> 172.21.15.123:0/3221544493 conn(0x7f8c78073200 msgr2=0x7f8c78073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 -- 172.21.15.123:0/3221544493 shutdown_connections 2024-04-25T19:04:19.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.821+0000 7f8c7f000640 1 -- 172.21.15.123:0/3221544493 wait complete. 2024-04-25T19:04:19.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.824+0000 7f8c7f000640 1 Processor -- start 2024-04-25T19:04:19.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.824+0000 7f8c7f000640 1 -- start start 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7806fe20 0x7f8c78144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7815e2f0 0x7f8c78144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7815edc0 con 0x7f8c7806d580 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7815ec40 con 0x7f8c7815e2f0 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7f000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c780ce500 con 0x7f8c7806fe20 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55662/0 (socket says 172.21.15.123:55662) 2024-04-25T19:04:19.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.826+0000 7f8c7dc00640 1 -- 172.21.15.123:0/1237978540 learned_addr learned my addr 172.21.15.123:0/1237978540 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7d200640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7806fe20 0x7f8c78144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7e600640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7815e2f0 0x7f8c78144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 -- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7806fe20 msgr2=0x7f8c78144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7806fe20 0x7f8c78144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 -- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7815e2f0 msgr2=0x7f8c78144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7815e2f0 0x7f8c78144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 -- 172.21.15.123:0/1237978540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c780cb130 con 0x7f8c7806d580 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.827+0000 7f8c7dc00640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c3c0 tx=0x7f8c6c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.828+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0052d0 con 0x7f8c7806d580 2024-04-25T19:04:19.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.828+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c005490 con 0x7f8c7806d580 2024-04-25T19:04:19.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.828+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c780cb410 con 0x7f8c7806d580 2024-04-25T19:04:19.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.828+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c007260 con 0x7f8c7806d580 2024-04-25T19:04:19.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.828+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c780cb9d0 con 0x7f8c7806d580 2024-04-25T19:04:19.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.829+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c78147780 con 0x7f8c7806d580 2024-04-25T19:04:19.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.830+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8c6c007400 con 0x7f8c7806d580 2024-04-25T19:04:19.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.831+0000 7f8c75600640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 0x7f8c580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.831+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8c6c0944b0 con 0x7f8c7806d580 2024-04-25T19:04:19.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.832+0000 7f8c7d200640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 0x7f8c580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.832+0000 7f8c7d200640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 0x7f8c580769f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c64034000 tx=0x7f8c6402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:19.836+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c7806ded0 con 0x7f8c7806d580 2024-04-25T19:04:20.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.036+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c78070a00 con 0x7f8c7806d580 2024-04-25T19:04:20.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.038+0000 7f8c75600640 1 -- 172.21.15.123:0/1237978540 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13212 (secure 0 0 0) 0x7f8c6c05ee90 con 0x7f8c7806d580 2024-04-25T19:04:20.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:20.040 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":36,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:04:15.427682+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.040+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 msgr2=0x7f8c580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.040+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 0x7f8c580769f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c64034000 tx=0x7f8c6402e000 comp rx=0 tx=0).stop 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.040+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 msgr2=0x7f8c780b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.040+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c3c0 tx=0x7f8c6c00c7c0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 shutdown_connections 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c580744e0 0x7f8c580769f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c7815e2f0 0x7f8c78144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c7806fe20 0x7f8c78144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 --2- 172.21.15.123:0/1237978540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c7806d580 0x7f8c780b58a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 >> 172.21.15.123:0/1237978540 conn(0x7f8c78073200 msgr2=0x7f8c78160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:20.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 shutdown_connections 2024-04-25T19:04:20.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.041+0000 7f8c7f000640 1 -- 172.21.15.123:0/1237978540 wait complete. 2024-04-25T19:04:20.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:04:20.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:04:20.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.332+0000 7f8091600640 1 Processor -- start 2024-04-25T19:04:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.333+0000 7f8091600640 1 -- start start 2024-04-25T19:04:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.334+0000 7f8091600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f808c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.334+0000 7f8091600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c06ded0 con 0x7f808c06dac0 2024-04-25T19:04:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.334+0000 7f8091600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c0d05f0 con 0x7f808c0cfff0 2024-04-25T19:04:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.334+0000 7f8091600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0d1200 con 0x7f808c143040 2024-04-25T19:04:20.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.335+0000 7f8090c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f808c06dac0 0x7f808c0cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43742/0 (socket says 172.21.15.123:43742) 2024-04-25T19:04:20.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.335+0000 7f8090c00640 1 -- 172.21.15.123:0/1672736219 learned_addr learned my addr 172.21.15.123:0/1672736219 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.335+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918884206 0 0) 0x7f8084001490 con 0x7f808c06dac0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.335+0000 7f8083400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f808c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.335+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8070003520 con 0x7f808c06dac0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1434051931 0 0) 0x7f8070003520 con 0x7f808c06dac0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 msgr2=0x7f808c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f808c06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 >> v1:172.21.15.146:6789/0 conn(0x7f808c0cfff0 legacy=0x7f808c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c0d08d0 con 0x7f808c06dac0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f8084001bc0 con 0x7f808c06dac0 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 >> v1:172.21.15.123:6789/0 conn(0x7f808c06dac0 legacy=0x7f808c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f80700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070004010 0x7f8070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f8070005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0d1200 con 0x7f8070004d40 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f807000de60 con 0x7f8070004010 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f807000e060 con 0x7f808c143040 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f8090c00640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f8070005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f808be00640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f80700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f8090c00640 1 -- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 msgr2=0x7f80700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f8090c00640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c143040 0x7f80700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f8090c00640 1 -- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070004010 msgr2=0x7f8070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.336+0000 7f8083400640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070004010 0x7f8070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.337+0000 7f8090c00640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070004010 0x7f8070004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.337+0000 7f8090c00640 1 -- 172.21.15.123:0/1672736219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8070003520 con 0x7f8070004d40 2024-04-25T19:04:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.338+0000 7f8090c00640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f8070005210 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f808400a9f0 tx=0x7f8070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7fffae52d9952c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.338+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808400a640 con 0x7f8070004d40 2024-04-25T19:04:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.338+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808402f070 con 0x7f8070004d40 2024-04-25T19:04:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.338+0000 7f808b400640 1 -- 172.21.15.123:0/1672736219 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084034470 con 0x7f8070004d40 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 -- 172.21.15.123:0/1672736219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 msgr2=0x7f8070005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 --2- 172.21.15.123:0/1672736219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f8070005210 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f808400a9f0 tx=0x7f8070005830 comp rx=0 tx=0).stop 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8090c00640 1 -- 172.21.15.123:0/1672736219 reap_dead start 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 -- 172.21.15.123:0/1672736219 shutdown_connections 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 -- 172.21.15.123:0/1672736219 >> 172.21.15.123:0/1672736219 conn(0x7f808c073090 msgr2=0x7f808c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 -- 172.21.15.123:0/1672736219 shutdown_connections 2024-04-25T19:04:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.339+0000 7f8091600640 1 -- 172.21.15.123:0/1672736219 wait complete. 2024-04-25T19:04:20.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.341+0000 7f8091600640 1 Processor -- start 2024-04-25T19:04:20.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.342+0000 7f8091600640 1 -- start start 2024-04-25T19:04:20.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 0x7f808c15f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c0b4a30 0x7f808c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f808c143450 con 0x7f8070004d40 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0d0e90 con 0x7f808c15f070 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f8091600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f808c138710 con 0x7f808c0b4a30 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f808be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f808be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55682/0 (socket says 172.21.15.123:55682) 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.343+0000 7f808be00640 1 -- 172.21.15.123:0/4041408479 learned_addr learned my addr 172.21.15.123:0/4041408479 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8083400640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 0x7f808c15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8090c00640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c0b4a30 0x7f808c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 -- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c0b4a30 msgr2=0x7f808c0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c0b4a30 0x7f808c0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 -- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 msgr2=0x7f808c15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 0x7f808c15f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 -- 172.21.15.123:0/4041408479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c0b5380 con 0x7f8070004d40 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8083400640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 0x7f808c15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:20.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f808be00640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e580 tx=0x7f807c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0052e0 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f807c018070 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c003810 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c15dd80 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.344+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c15e340 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.346+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c050740 con 0x7f8070004d40 2024-04-25T19:04:20.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.347+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f807c0039b0 con 0x7f8070004d40 2024-04-25T19:04:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.348+0000 7f8088c00640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 0x7f80680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.348+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f807c094e70 con 0x7f8070004d40 2024-04-25T19:04:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.348+0000 7f8083400640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 0x7f80680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.349+0000 7f8083400640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 0x7f80680769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f808c068850 tx=0x7f8074008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.353+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f808c0b5380 con 0x7f8070004d40 2024-04-25T19:04:20.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.540+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f808c139e20 con 0x7f8070004d40 2024-04-25T19:04:20.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.541+0000 7f8088c00640 1 -- 172.21.15.123:0/4041408479 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f807c05f7c0 con 0x7f8070004d40 2024-04-25T19:04:20.543 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 msgr2=0x7f80680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 0x7f80680769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f808c068850 tx=0x7f8074008040 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 msgr2=0x7f808c154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e580 tx=0x7f807c00ea40 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 shutdown_connections 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80680744e0 0x7f80680769f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f808c0b4a30 0x7f808c0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808c15f070 0x7f808c15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 --2- 172.21.15.123:0/4041408479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8070004d40 0x7f808c154a50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 >> 172.21.15.123:0/4041408479 conn(0x7f808c073090 msgr2=0x7f808c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 shutdown_connections 2024-04-25T19:04:20.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.544+0000 7f8091600640 1 -- 172.21.15.123:0/4041408479 wait complete. 2024-04-25T19:04:20.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T19:04:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.814+0000 7f5685400640 1 Processor -- start 2024-04-25T19:04:20.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.814+0000 7f5685400640 1 -- start start 2024-04-25T19:04:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5685400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f56800d3d30 0x7f56800b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5685400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5680160da0 0x7f568006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5685400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56800b51c0 con 0x7f56800d4520 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5685400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f56800d4140 con 0x7f5680160da0 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5685400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f568006c7a0 con 0x7f56800d3d30 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5684a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5680160da0 0x7f568006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5677400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f56800d4520 0x7f5680160690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43748/0 (socket says 172.21.15.123:43748) 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f5677400640 1 -- 172.21.15.123:0/280724692 learned_addr learned my addr 172.21.15.123:0/280724692 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.816+0000 7f567fe00640 1 --2- 172.21.15.123:0/280724692 >> v2:172.21.15.146:3300/0 conn(0x7f56800d3d30 0x7f56800b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.817+0000 7f5684a00640 1 -- 172.21.15.123:0/280724692 >> v2:172.21.15.146:3300/0 conn(0x7f56800d3d30 msgr2=0x7f56800b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.817+0000 7f5684a00640 1 --2- 172.21.15.123:0/280724692 >> v2:172.21.15.146:3300/0 conn(0x7f56800d3d30 0x7f56800b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.817+0000 7f5684a00640 1 -- 172.21.15.123:0/280724692 >> v1:172.21.15.123:6789/0 conn(0x7f56800d4520 legacy=0x7f5680160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.817+0000 7f5684a00640 1 -- 172.21.15.123:0/280724692 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56800d4d00 con 0x7f5680160da0 2024-04-25T19:04:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.818+0000 7f5684a00640 1 --2- 172.21.15.123:0/280724692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5680160da0 0x7f568006cc30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f568008e670 tx=0x7f567802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118ca6b00bf49287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.818+0000 7f567f400640 1 -- 172.21.15.123:0/280724692 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678007740 con 0x7f5680160da0 2024-04-25T19:04:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.818+0000 7f567f400640 1 -- 172.21.15.123:0/280724692 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56780078e0 con 0x7f5680160da0 2024-04-25T19:04:20.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.818+0000 7f567f400640 1 -- 172.21.15.123:0/280724692 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678007cb0 con 0x7f5680160da0 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 -- 172.21.15.123:0/280724692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5680160da0 msgr2=0x7f568006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 --2- 172.21.15.123:0/280724692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5680160da0 0x7f568006cc30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f568008e670 tx=0x7f567802a680 comp rx=0 tx=0).stop 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 -- 172.21.15.123:0/280724692 shutdown_connections 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 --2- 172.21.15.123:0/280724692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5680160da0 0x7f568006cc30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 --2- 172.21.15.123:0/280724692 >> v2:172.21.15.146:3300/0 conn(0x7f56800d3d30 0x7f56800b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 -- 172.21.15.123:0/280724692 >> 172.21.15.123:0/280724692 conn(0x7f56800b7440 msgr2=0x7f56800b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 -- 172.21.15.123:0/280724692 shutdown_connections 2024-04-25T19:04:20.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.819+0000 7f5685400640 1 -- 172.21.15.123:0/280724692 wait complete. 2024-04-25T19:04:20.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.822+0000 7f5685400640 1 Processor -- start 2024-04-25T19:04:20.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.822+0000 7f5685400640 1 -- start start 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56800d4520 0x7f56800caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680160da0 0x7f56801f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56800d4930 con 0x7f56800d3d30 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56800d52c0 con 0x7f5680160da0 2024-04-25T19:04:20.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f5685400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f56800d4ab0 con 0x7f56800d4520 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55694/0 (socket says 172.21.15.123:55694) 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 -- 172.21.15.123:0/3360141056 learned_addr learned my addr 172.21.15.123:0/3360141056 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 -- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56800d4520 msgr2=0x7f56800caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56800d4520 0x7f56800caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 -- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680160da0 msgr2=0x7f56801f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680160da0 0x7f56801f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:20.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.824+0000 7f567fe00640 1 -- 172.21.15.123:0/3360141056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5680156b40 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f567fe00640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f567000d5f0 tx=0x7f567000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700125f0 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670019070 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5680157070 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700129d0 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.825+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5680157630 con 0x7f56800d3d30 2024-04-25T19:04:20.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.826+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5680050740 con 0x7f56800d3d30 2024-04-25T19:04:20.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.827+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5670012b70 con 0x7f56800d3d30 2024-04-25T19:04:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.828+0000 7f567cc00640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 0x7f565c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.828+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5670095540 con 0x7f56800d3d30 2024-04-25T19:04:20.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.829+0000 7f5677400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 0x7f565c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:20.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.829+0000 7f5677400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 0x7f565c0769c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5668000f60 tx=0x7f5668007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:20.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:20.833+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f567005fdf0 con 0x7f56800d3d30 2024-04-25T19:04:21.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.020+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f56800b56d0 con 0x7f56800d3d30 2024-04-25T19:04:21.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.021+0000 7f567cc00640 1 -- 172.21.15.123:0/3360141056 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f5670066e20 con 0x7f56800d3d30 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-25T19:04:21.025 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-25T19:04:21.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 msgr2=0x7f565c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 0x7f565c0769c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5668000f60 tx=0x7f5668007a80 comp rx=0 tx=0).stop 2024-04-25T19:04:21.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 msgr2=0x7f5680158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f567000d5f0 tx=0x7f567000dab0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 shutdown_connections 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f565c0744b0 0x7f565c0769c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680160da0 0x7f56801f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56800d4520 0x7f56800caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 --2- 172.21.15.123:0/3360141056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56800d3d30 0x7f5680158110 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 >> 172.21.15.123:0/3360141056 conn(0x7f56800b7440 msgr2=0x7f568015f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.024+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 shutdown_connections 2024-04-25T19:04:21.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.025+0000 7f5685400640 1 -- 172.21.15.123:0/3360141056 wait complete. 2024-04-25T19:04:21.050 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T19:04:21.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T19:04:21.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.315+0000 7f8f7ac00640 1 Processor -- start 2024-04-25T19:04:21.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.316+0000 7f8f7ac00640 1 -- start start 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f7ac00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f741441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f7ac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f740cdce0 con 0x7f8f74147200 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f7ac00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f740cdf60 con 0x7f8f7406dc90 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f7ac00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f740b5100 con 0x7f8f740cd8d0 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f79800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8f74147200 0x7f8f7406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43752/0 (socket says 172.21.15.123:43752) 2024-04-25T19:04:21.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.318+0000 7f8f79800640 1 -- 172.21.15.123:0/3284200461 learned_addr learned my addr 172.21.15.123:0/3284200461 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676774890 0 0) 0x7f8f68001110 con 0x7f8f74147200 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f5c003520 con 0x7f8f74147200 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f7a200640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f741441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1413566569 0 0) 0x7f8f5c003520 con 0x7f8f74147200 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 msgr2=0x7f8f741441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:21.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f741441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 >> v1:172.21.15.146:6789/0 conn(0x7f8f7406dc90 legacy=0x7f8f7406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f740b4810 con 0x7f8f74147200 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f8f68001840 con 0x7f8f74147200 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 >> v1:172.21.15.123:6789/0 conn(0x7f8f74147200 legacy=0x7f8f7406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f5c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740b5100 con 0x7f8f5c0039e0 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c00bc60 con 0x7f8f5c004d90 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c00be60 con 0x7f8f740cd8d0 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.319+0000 7f8f79800640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f5c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f78e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f7a200640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f79800640 1 -- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 msgr2=0x7f8f5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f79800640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f79800640 1 -- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f5c004d90 msgr2=0x7f8f5c005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f79800640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f79800640 1 -- 172.21.15.123:0/3284200461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c003520 con 0x7f8f5c0039e0 2024-04-25T19:04:21.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.320+0000 7f8f78e00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f740cd8d0 0x7f8f5c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.321+0000 7f8f79800640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f5c003e30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c320 tx=0x7f8f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ae93a1e404f8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.321+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68005ad0 con 0x7f8f5c0039e0 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.321+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68034070 con 0x7f8f5c0039e0 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.321+0000 7f8f73e00640 1 -- 172.21.15.123:0/3284200461 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68007070 con 0x7f8f5c0039e0 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 -- 172.21.15.123:0/3284200461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 msgr2=0x7f8f5c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/3284200461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f5c003e30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c320 tx=0x7f8f5c005830 comp rx=0 tx=0).stop 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7a200640 1 -- 172.21.15.123:0/3284200461 reap_dead start 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 -- 172.21.15.123:0/3284200461 shutdown_connections 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 -- 172.21.15.123:0/3284200461 >> 172.21.15.123:0/3284200461 conn(0x7f8f74072fd0 msgr2=0x7f8f740733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 -- 172.21.15.123:0/3284200461 shutdown_connections 2024-04-25T19:04:21.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.322+0000 7f8f7ac00640 1 -- 172.21.15.123:0/3284200461 wait complete. 2024-04-25T19:04:21.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.324+0000 7f8f7ac00640 1 Processor -- start 2024-04-25T19:04:21.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.325+0000 7f8f7ac00640 1 -- start start 2024-04-25T19:04:21.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 0x7f8f7413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f7413fce0 0x7f8f7413b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74147a10 con 0x7f8f5c0039e0 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740b4d90 con 0x7f8f7413fce0 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f7ac00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f740b5590 con 0x7f8f7415d380 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f79800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f79800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55714/0 (socket says 172.21.15.123:55714) 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.326+0000 7f8f79800640 1 -- 172.21.15.123:0/2572136531 learned_addr learned my addr 172.21.15.123:0/2572136531 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f78e00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 0x7f8f7413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f7a200640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f7413fce0 0x7f8f7413b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 -- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 msgr2=0x7f8f7413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 0x7f8f7413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 -- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f7413fce0 msgr2=0x7f8f7413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f7413fce0 0x7f8f7413b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 -- 172.21.15.123:0/2572136531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f74138e50 con 0x7f8f5c0039e0 2024-04-25T19:04:21.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f78e00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 0x7f8f7413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:21.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.327+0000 7f8f79800640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f6802af70 tx=0x7f8f68001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.328+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68007270 con 0x7f8f5c0039e0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.328+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68007e50 con 0x7f8f5c0039e0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.328+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68007650 con 0x7f8f5c0039e0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.328+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f74139130 con 0x7f8f5c0039e0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.328+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741396f0 con 0x7f8f5c0039e0 2024-04-25T19:04:21.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.329+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f74073d10 con 0x7f8f5c0039e0 2024-04-25T19:04:21.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.330+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f6803a410 con 0x7f8f5c0039e0 2024-04-25T19:04:21.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.331+0000 7f8f71600640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 0x7f8f540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:21.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.331+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8f680b5830 con 0x7f8f5c0039e0 2024-04-25T19:04:21.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.332+0000 7f8f78e00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 0x7f8f540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:21.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.332+0000 7f8f78e00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 0x7f8f540769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8f60003dc0 tx=0x7f8f60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:21.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.337+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f740cdce0 con 0x7f8f5c0039e0 2024-04-25T19:04:21.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T19:04:21.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T19:04:21.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.536+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8f74073d10 con 0x7f8f5c0039e0 2024-04-25T19:04:21.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.987+0000 7f8f71600640 1 -- 172.21.15.123:0/2572136531 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f8f68080160 con 0x7f8f5c0039e0 2024-04-25T19:04:21.990 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-25T19:04:21.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.990+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 msgr2=0x7f8f540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 0x7f8f540769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8f60003dc0 tx=0x7f8f60009040 comp rx=0 tx=0).stop 2024-04-25T19:04:21.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 msgr2=0x7f8f74139ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:21.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f6802af70 tx=0x7f8f68001b40 comp rx=0 tx=0).stop 2024-04-25T19:04:21.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 shutdown_connections 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8f540744e0 0x7f8f540769f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f7413fce0 0x7f8f7413b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f7415d380 0x7f8f7413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 --2- 172.21.15.123:0/2572136531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f5c0039e0 0x7f8f74139ef0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.991+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 >> 172.21.15.123:0/2572136531 conn(0x7f8f74072fd0 msgr2=0x7f8f740b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.992+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 shutdown_connections 2024-04-25T19:04:21.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:21.992+0000 7f8f7ac00640 1 -- 172.21.15.123:0/2572136531 wait complete. 2024-04-25T19:04:22.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T19:04:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.284+0000 7f242c000640 1 Processor -- start 2024-04-25T19:04:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.284+0000 7f242c000640 1 -- start start 2024-04-25T19:04:22.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.286+0000 7f242c000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24241601c0 0x7f24240cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.286+0000 7f242c000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240ccd00 con 0x7f2424070120 2024-04-25T19:04:22.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f242c000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240ccfe0 con 0x7f24240d06c0 2024-04-25T19:04:22.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f242c000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f24240cfd80 con 0x7f24241601c0 2024-04-25T19:04:22.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f2429600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2424070120 0x7f24240cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43758/0 (socket says 172.21.15.123:43758) 2024-04-25T19:04:22.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f2429600640 1 -- 172.21.15.123:0/648403506 learned_addr learned my addr 172.21.15.123:0/648403506 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124256265 0 0) 0x7f24240ccd00 con 0x7f2424070120 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2408003520 con 0x7f2424070120 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.287+0000 7f242a000640 1 --2- 172.21.15.123:0/648403506 >> v2:172.21.15.146:3300/0 conn(0x7f24241601c0 0x7f24240cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2809403740 0 0) 0x7f2408003520 con 0x7f2424070120 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 >> v2:172.21.15.146:3300/0 conn(0x7f24241601c0 msgr2=0x7f24240cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 --2- 172.21.15.123:0/648403506 >> v2:172.21.15.146:3300/0 conn(0x7f24241601c0 0x7f24240cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 >> v1:172.21.15.146:6789/0 conn(0x7f24240d06c0 legacy=0x7f242406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240cd2c0 con 0x7f2424070120 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f2414001670 con 0x7f2424070120 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 >> v1:172.21.15.123:6789/0 conn(0x7f2424070120 legacy=0x7f24240cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f2408003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2408004800 0x7f2408004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24080051c0 0x7f240800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f240800c130 con 0x7f24080039b0 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f240800c2e0 con 0x7f24080051c0 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f240800c4e0 con 0x7f2408004800 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.288+0000 7f2429600640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f2408003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2428c00640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2408004800 0x7f2408004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2429600640 1 -- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2408004800 msgr2=0x7f2408004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2429600640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2408004800 0x7f2408004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2429600640 1 -- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24080051c0 msgr2=0x7f240800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2429600640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24080051c0 0x7f240800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f2429600640 1 -- 172.21.15.123:0/648403506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2408003520 con 0x7f24080039b0 2024-04-25T19:04:22.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.289+0000 7f242a000640 1 -- 172.21.15.123:0/648403506 reap_dead start 2024-04-25T19:04:22.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.290+0000 7f2429600640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f2408003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f241400c210 tx=0x7f2408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34a72946c29e186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:22.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.290+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414007d10 con 0x7f24080039b0 2024-04-25T19:04:22.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.290+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414005ad0 con 0x7f24080039b0 2024-04-25T19:04:22.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.290+0000 7f2423e00640 1 -- 172.21.15.123:0/648403506 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414007070 con 0x7f24080039b0 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 -- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 msgr2=0x7f2408003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f2408003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f241400c210 tx=0x7f2408005830 comp rx=0 tx=0).stop 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 -- 172.21.15.123:0/648403506 shutdown_connections 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 --2- 172.21.15.123:0/648403506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f2408003e20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 -- 172.21.15.123:0/648403506 >> 172.21.15.123:0/648403506 conn(0x7f24240ce390 msgr2=0x7f24240ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 -- 172.21.15.123:0/648403506 shutdown_connections 2024-04-25T19:04:22.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.291+0000 7f242c000640 1 -- 172.21.15.123:0/648403506 wait complete. 2024-04-25T19:04:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.293+0000 7f242c000640 1 Processor -- start 2024-04-25T19:04:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.294+0000 7f242c000640 1 -- start start 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f242414b100 0x7f24241f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24241f65a0 0x7f24241f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24240cd910 con 0x7f24080039b0 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24240cdda0 con 0x7f242414b100 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f242c000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f24240cc570 con 0x7f24241f65a0 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f2429600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f2429600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55730/0 (socket says 172.21.15.123:55730) 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.295+0000 7f2429600640 1 -- 172.21.15.123:0/3858390742 learned_addr learned my addr 172.21.15.123:0/3858390742 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2428c00640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f242414b100 0x7f24241f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f242a000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24241f65a0 0x7f24241f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 -- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24241f65a0 msgr2=0x7f24241f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24241f65a0 0x7f24241f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 -- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f242414b100 msgr2=0x7f24241f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f242414b100 0x7f24241f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 -- 172.21.15.123:0/3858390742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242414ae30 con 0x7f24080039b0 2024-04-25T19:04:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.296+0000 7f2429600640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f241400c270 tx=0x7f24240cd1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.297+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241403f020 con 0x7f24080039b0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.297+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414001430 con 0x7f24080039b0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.297+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140047e0 con 0x7f24080039b0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.297+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24240b5750 con 0x7f24080039b0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.297+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24240b5d10 con 0x7f24080039b0 2024-04-25T19:04:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.298+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2424076a40 con 0x7f24080039b0 2024-04-25T19:04:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.300+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2414042020 con 0x7f24080039b0 2024-04-25T19:04:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.302+0000 7f2421600640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 0x7f24040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.302+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f24140b5430 con 0x7f24080039b0 2024-04-25T19:04:22.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.302+0000 7f2428c00640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 0x7f24040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:22.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.303+0000 7f2428c00640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 0x7f24040769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24240b0820 tx=0x7f2410009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:22.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.306+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f241407fce0 con 0x7f24080039b0 2024-04-25T19:04:22.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.506+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2424069c60 con 0x7f24080039b0 2024-04-25T19:04:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.995+0000 7f2421600640 1 -- 172.21.15.123:0/3858390742 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f241407f5e0 con 0x7f24080039b0 2024-04-25T19:04:22.997 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 msgr2=0x7f24040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 0x7f24040769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24240b0820 tx=0x7f2410009040 comp rx=0 tx=0).stop 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 msgr2=0x7f242414a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f241400c270 tx=0x7f24240cd1b0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 shutdown_connections 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f24040744e0 0x7f24040769f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24241f65a0 0x7f24241f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f242414b100 0x7f24241f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 --2- 172.21.15.123:0/3858390742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24080039b0 0x7f242414a8f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 >> 172.21.15.123:0/3858390742 conn(0x7f24240ce390 msgr2=0x7f24240b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 shutdown_connections 2024-04-25T19:04:23.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:22.999+0000 7f242c000640 1 -- 172.21.15.123:0/3858390742 wait complete. 2024-04-25T19:04:23.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T19:04:23.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:23.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:23.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.295+0000 7f2700c00640 1 Processor -- start 2024-04-25T19:04:23.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.296+0000 7f2700c00640 1 -- start start 2024-04-25T19:04:23.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f2700c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc15df10 0x7f26fc0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f2700c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26fc06d580 0x7f26fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f2700c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc0d1430 con 0x7f26fc0b0700 2024-04-25T19:04:23.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f2700c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f26fc070f40 con 0x7f26fc06d580 2024-04-25T19:04:23.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f2700c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc161e90 con 0x7f26fc15df10 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f26fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26fc06d580 0x7f26fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f26f2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f26fc0b0700 0x7f26fc162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43762/0 (socket says 172.21.15.123:43762) 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f26fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26fc06d580 0x7f26fc06d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55732/0 (socket says 172.21.15.123:55732) 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.298+0000 7f26f2a00640 1 -- 172.21.15.123:0/2231077863 learned_addr learned my addr 172.21.15.123:0/2231077863 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26fb400640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc15df10 0x7f26fc0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26fbe00640 1 -- 172.21.15.123:0/2231077863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc15df10 msgr2=0x7f26fc0702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26fbe00640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc15df10 0x7f26fc0702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26fbe00640 1 -- 172.21.15.123:0/2231077863 >> v1:172.21.15.123:6789/0 conn(0x7f26fc0b0700 legacy=0x7f26fc162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26fbe00640 1 -- 172.21.15.123:0/2231077863 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc06ded0 con 0x7f26fc06d580 2024-04-25T19:04:23.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.299+0000 7f26faa00640 1 -- 172.21.15.123:0/2231077863 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039026742 0 0) 0x7f26fc0d1430 con 0x7f26fc0b0700 2024-04-25T19:04:23.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.300+0000 7f26fbe00640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc06d580 0x7f26fc06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f26f40097a0 tx=0x7f26f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6b7a45a2a5942d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:23.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.301+0000 7f26faa00640 1 -- 172.21.15.123:0/2231077863 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4037070 con 0x7f26fc06d580 2024-04-25T19:04:23.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.301+0000 7f26faa00640 1 -- 172.21.15.123:0/2231077863 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f4032440 con 0x7f26fc06d580 2024-04-25T19:04:23.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.301+0000 7f26faa00640 1 -- 172.21.15.123:0/2231077863 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4032810 con 0x7f26fc06d580 2024-04-25T19:04:23.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 -- 172.21.15.123:0/2231077863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc06d580 msgr2=0x7f26fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc06d580 0x7f26fc06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f26f40097a0 tx=0x7f26f402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 -- 172.21.15.123:0/2231077863 shutdown_connections 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc06d580 0x7f26fc06d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 --2- 172.21.15.123:0/2231077863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc15df10 0x7f26fc0702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 -- 172.21.15.123:0/2231077863 >> 172.21.15.123:0/2231077863 conn(0x7f26fc073090 msgr2=0x7f26fc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 -- 172.21.15.123:0/2231077863 shutdown_connections 2024-04-25T19:04:23.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.302+0000 7f2700c00640 1 -- 172.21.15.123:0/2231077863 wait complete. 2024-04-25T19:04:23.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.304+0000 7f2700c00640 1 Processor -- start 2024-04-25T19:04:23.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.304+0000 7f2700c00640 1 -- start start 2024-04-25T19:04:23.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26fc06d580 0x7f26fc14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc0b0700 0x7f26fc15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc161b20 con 0x7f26fc15df10 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc070c60 con 0x7f26fc06d580 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f2700c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc0cdfe0 con 0x7f26fc0b0700 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f26fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f26fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55734/0 (socket says 172.21.15.123:55734) 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.306+0000 7f26fbe00640 1 -- 172.21.15.123:0/483364099 learned_addr learned my addr 172.21.15.123:0/483364099 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26f2a00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc0b0700 0x7f26fc15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fb400640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26fc06d580 0x7f26fc14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 -- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc0b0700 msgr2=0x7f26fc15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc0b0700 0x7f26fc15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 -- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26fc06d580 msgr2=0x7f26fc14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26fc06d580 0x7f26fc14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 -- 172.21.15.123:0/483364099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc151430 con 0x7f26fc15df10 2024-04-25T19:04:23.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.307+0000 7f26fbe00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26f40097a0 tx=0x7f26f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.308+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4037050 con 0x7f26fc15df10 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.308+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f4038040 con 0x7f26fc15df10 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.308+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004270 con 0x7f26fc15df10 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.308+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc151710 con 0x7f26fc15df10 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.308+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc151cd0 con 0x7f26fc15df10 2024-04-25T19:04:23.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.309+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc050740 con 0x7f26fc15df10 2024-04-25T19:04:23.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.311+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26f4030980 con 0x7f26fc15df10 2024-04-25T19:04:23.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.312+0000 7f26f3e00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 0x7f26d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:23.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.312+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f26f40b6ce0 con 0x7f26fc15df10 2024-04-25T19:04:23.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.312+0000 7f26fb400640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 0x7f26d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.313+0000 7f26fb400640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 0x7f26d80769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26fc1618e0 tx=0x7f26ec003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:23.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.316+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26fc06ded0 con 0x7f26fc15df10 2024-04-25T19:04:23.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:23.544+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f26fc15fb00 con 0x7f26fc15df10 2024-04-25T19:04:24.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.045+0000 7f26f3e00640 1 -- 172.21.15.123:0/483364099 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f26f4081480 con 0x7f26fc15df10 2024-04-25T19:04:24.047 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 msgr2=0x7f26d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 0x7f26d80769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26fc1618e0 tx=0x7f26ec003990 comp rx=0 tx=0).stop 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 msgr2=0x7f26fc15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26f40097a0 tx=0x7f26f40046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 shutdown_connections 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f26d80744b0 0x7f26d80769c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26fc15df10 0x7f26fc15f560 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26fc0b0700 0x7f26fc15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 --2- 172.21.15.123:0/483364099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26fc06d580 0x7f26fc14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.051+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 >> 172.21.15.123:0/483364099 conn(0x7f26fc073090 msgr2=0x7f26fc0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:24.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.052+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 shutdown_connections 2024-04-25T19:04:24.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.052+0000 7f2700c00640 1 -- 172.21.15.123:0/483364099 wait complete. 2024-04-25T19:04:24.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:24.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.330+0000 7f67ec600640 1 Processor -- start 2024-04-25T19:04:24.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.330+0000 7f67ec600640 1 -- start start 2024-04-25T19:04:24.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67ec600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67e415df00 0x7f67e406f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67ec600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e40b4750 con 0x7f67e406f780 2024-04-25T19:04:24.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67ec600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e40b4a30 con 0x7f67e40b5f00 2024-04-25T19:04:24.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67ec600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f67e415e310 con 0x7f67e415df00 2024-04-25T19:04:24.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67e9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67e415df00 0x7f67e406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67e9200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f67e406f780 0x7f67e40b57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43766/0 (socket says 172.21.15.123:43766) 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.333+0000 7f67e9200640 1 -- 172.21.15.123:0/627449901 learned_addr learned my addr 172.21.15.123:0/627449901 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.334+0000 7f67e9c00640 1 -- 172.21.15.123:0/627449901 >> v1:172.21.15.146:6789/0 conn(0x7f67e40b5f00 legacy=0x7f67e40ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.334+0000 7f67e9c00640 1 -- 172.21.15.123:0/627449901 >> v1:172.21.15.123:6789/0 conn(0x7f67e406f780 legacy=0x7f67e40b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.334+0000 7f67e3e00640 1 -- 172.21.15.123:0/627449901 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949387623 0 0) 0x7f67e40b4750 con 0x7f67e406f780 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.334+0000 7f67e9c00640 1 -- 172.21.15.123:0/627449901 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e40b4d10 con 0x7f67e415df00 2024-04-25T19:04:24.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.335+0000 7f67e9c00640 1 --2- 172.21.15.123:0/627449901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e406f240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097a0 tx=0x7f67d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948de51b8d43b05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.335+0000 7f67e3e00640 1 -- 172.21.15.123:0/627449901 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8037070 con 0x7f67e415df00 2024-04-25T19:04:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.335+0000 7f67e3e00640 1 -- 172.21.15.123:0/627449901 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d802ae50 con 0x7f67e415df00 2024-04-25T19:04:24.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.335+0000 7f67e3e00640 1 -- 172.21.15.123:0/627449901 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8032660 con 0x7f67e415df00 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 -- 172.21.15.123:0/627449901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 msgr2=0x7f67e406f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 --2- 172.21.15.123:0/627449901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e406f240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097a0 tx=0x7f67d802a140 comp rx=0 tx=0).stop 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 -- 172.21.15.123:0/627449901 shutdown_connections 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 --2- 172.21.15.123:0/627449901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e406f240 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 -- 172.21.15.123:0/627449901 >> 172.21.15.123:0/627449901 conn(0x7f67e40d1ae0 msgr2=0x7f67e40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 -- 172.21.15.123:0/627449901 shutdown_connections 2024-04-25T19:04:24.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.336+0000 7f67ec600640 1 -- 172.21.15.123:0/627449901 wait complete. 2024-04-25T19:04:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.338+0000 7f67ec600640 1 Processor -- start 2024-04-25T19:04:24.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.339+0000 7f67ec600640 1 -- start start 2024-04-25T19:04:24.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67e406f780 0x7f67e414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67e40b5f00 0x7f67e41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40aeef0 con 0x7f67e415df00 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40b5360 con 0x7f67e40b5f00 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.340+0000 7f67ec600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f67e40af070 con 0x7f67e406f780 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55750/0 (socket says 172.21.15.123:55750) 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 -- 172.21.15.123:0/654498468 learned_addr learned my addr 172.21.15.123:0/654498468 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67e9c00640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67e406f780 0x7f67e414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67e9200640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67e40b5f00 0x7f67e41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 -- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67e406f780 msgr2=0x7f67e414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67e406f780 0x7f67e414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 -- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67e40b5f00 msgr2=0x7f67e41573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67e40b5f00 0x7f67e41573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.341+0000 7f67ea600640 1 -- 172.21.15.123:0/654498468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e41fb8a0 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67ea600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b9e0 tx=0x7f67dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc003b70 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc003d10 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e41fb5c0 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00caa0 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.342+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e4156bb0 con 0x7f67e415df00 2024-04-25T19:04:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.343+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e4076a40 con 0x7f67e415df00 2024-04-25T19:04:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.345+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f67dc00cc40 con 0x7f67e415df00 2024-04-25T19:04:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.345+0000 7f67e1600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 0x7f67c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.346+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f67dc095150 con 0x7f67e415df00 2024-04-25T19:04:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.346+0000 7f67e9c00640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 0x7f67c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.347+0000 7f67e9c00640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 0x7f67c40769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67d8034000 tx=0x7f67d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.350+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67dc05f7e0 con 0x7f67e415df00 2024-04-25T19:04:24.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.539+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67e40b2c40 con 0x7f67e415df00 2024-04-25T19:04:24.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.542+0000 7f67e1600640 1 -- 172.21.15.123:0/654498468 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17003 (secure 0 0 0) 0x7f67dc05f0e0 con 0x7f67e415df00 2024-04-25T19:04:24.545 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:24.545 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":39,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:04:23.546086+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:24.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 msgr2=0x7f67c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 0x7f67c40769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67d8034000 tx=0x7f67d802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:24.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 msgr2=0x7f67e41fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b9e0 tx=0x7f67dc00bea0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 shutdown_connections 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f67c40744b0 0x7f67c40769c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e415df00 0x7f67e41fa900 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67e40b5f00 0x7f67e41573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 --2- 172.21.15.123:0/654498468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67e406f780 0x7f67e414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.545+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 >> 172.21.15.123:0/654498468 conn(0x7f67e40d1ae0 msgr2=0x7f67e415af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.546+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 shutdown_connections 2024-04-25T19:04:24.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.546+0000 7f67ec600640 1 -- 172.21.15.123:0/654498468 wait complete. 2024-04-25T19:04:24.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:24.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.840+0000 7fe8bd800640 1 Processor -- start 2024-04-25T19:04:24.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.841+0000 7fe8bd800640 1 -- start start 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8bd800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8b80d1120 0x7fe8b8070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8bd800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b8154950 0x7fe8b8154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8bd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b80aec30 con 0x7fe8b8161b10 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8bd800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b80d1530 con 0x7fe8b80d1120 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8bd800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b80aef10 con 0x7fe8b8154950 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8ae000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe8b8161b10 0x7fe8b8161f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43772/0 (socket says 172.21.15.123:43772) 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8ae000640 1 -- 172.21.15.123:0/1548628019 learned_addr learned my addr 172.21.15.123:0/1548628019 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.843+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/1548628019 >> v2:172.21.15.123:3300/0 conn(0x7fe8b80d1120 0x7fe8b8070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b7400640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b8154950 0x7fe8b8154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6000640 1 -- 172.21.15.123:0/1548628019 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264561637 0 0) 0x7fe8b80aec30 con 0x7fe8b8161b10 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6000640 1 -- 172.21.15.123:0/1548628019 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe89c003520 con 0x7fe8b8161b10 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6a00640 1 -- 172.21.15.123:0/1548628019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b8154950 msgr2=0x7fe8b8154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b8154950 0x7fe8b8154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6a00640 1 -- 172.21.15.123:0/1548628019 >> v1:172.21.15.123:6789/0 conn(0x7fe8b8161b10 legacy=0x7fe8b8161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.844+0000 7fe8b6a00640 1 -- 172.21.15.123:0/1548628019 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b80af1f0 con 0x7fe8b80d1120 2024-04-25T19:04:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.845+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b80d1120 0x7fe8b8070ef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097a0 tx=0x7fe8a802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a218ebc19e581f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.845+0000 7fe8b6000640 1 -- 172.21.15.123:0/1548628019 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8037070 con 0x7fe8b80d1120 2024-04-25T19:04:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.845+0000 7fe8b6000640 1 -- 172.21.15.123:0/1548628019 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a802adb0 con 0x7fe8b80d1120 2024-04-25T19:04:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.845+0000 7fe8b6000640 1 -- 172.21.15.123:0/1548628019 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8032660 con 0x7fe8b80d1120 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 -- 172.21.15.123:0/1548628019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b80d1120 msgr2=0x7fe8b8070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b80d1120 0x7fe8b8070ef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097a0 tx=0x7fe8a802a060 comp rx=0 tx=0).stop 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 -- 172.21.15.123:0/1548628019 shutdown_connections 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b8154950 0x7fe8b8154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 --2- 172.21.15.123:0/1548628019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b80d1120 0x7fe8b8070ef0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 -- 172.21.15.123:0/1548628019 >> 172.21.15.123:0/1548628019 conn(0x7fe8b80d1ae0 msgr2=0x7fe8b80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 -- 172.21.15.123:0/1548628019 shutdown_connections 2024-04-25T19:04:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.846+0000 7fe8bd800640 1 -- 172.21.15.123:0/1548628019 wait complete. 2024-04-25T19:04:24.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.849+0000 7fe8bd800640 1 Processor -- start 2024-04-25T19:04:24.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.849+0000 7fe8bd800640 1 -- start start 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b80d1120 0x7fe8b81f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 0x7fe8b81f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81552a0 con 0x7fe8b8154950 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8071430 con 0x7fe8b8161b10 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8bd800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b8156150 con 0x7fe8b80d1120 2024-04-25T19:04:24.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.850+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55774/0 (socket says 172.21.15.123:55774) 2024-04-25T19:04:24.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 -- 172.21.15.123:0/2435327926 learned_addr learned my addr 172.21.15.123:0/2435327926 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b80d1120 0x7fe8b81f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8b7400640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 0x7fe8b81f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 -- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b80d1120 msgr2=0x7fe8b81f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b80d1120 0x7fe8b81f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 -- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 msgr2=0x7fe8b81f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 0x7fe8b81f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 -- 172.21.15.123:0/2435327926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b81f5440 con 0x7fe8b8154950 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8b7400640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 0x7fe8b81f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.851+0000 7fe8ae000640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c3c0 tx=0x7fe8a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.852+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0017070 con 0x7fe8b8154950 2024-04-25T19:04:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.852+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a0005ce0 con 0x7fe8b8154950 2024-04-25T19:04:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.852+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00072f0 con 0x7fe8b8154950 2024-04-25T19:04:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.852+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b81f5720 con 0x7fe8b8154950 2024-04-25T19:04:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.852+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b8155e90 con 0x7fe8b8154950 2024-04-25T19:04:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.853+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b81f51b0 con 0x7fe8b8154950 2024-04-25T19:04:24.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.854+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8a001a070 con 0x7fe8b8154950 2024-04-25T19:04:24.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.855+0000 7fe8af400640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 0x7fe8940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:24.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.855+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe8a0094700 con 0x7fe8b8154950 2024-04-25T19:04:24.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.856+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 0x7fe8940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:24.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.856+0000 7fe8b6a00640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 0x7fe8940769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:24.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:24.860+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8a005ed90 con 0x7fe8b8154950 2024-04-25T19:04:25.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.069+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8b8076a40 con 0x7fe8b8154950 2024-04-25T19:04:25.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.072+0000 7fe8af400640 1 -- 172.21.15.123:0/2435327926 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16990 (secure 0 0 0) 0x7fe8a005e690 con 0x7fe8b8154950 2024-04-25T19:04:25.074 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:25.074 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:04:25.027549+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 msgr2=0x7fe8940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 0x7fe8940769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 msgr2=0x7fe8b814e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c3c0 tx=0x7fe8a000c880 comp rx=0 tx=0).stop 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 shutdown_connections 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe8940744e0 0x7fe8940769f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8161b10 0x7fe8b81f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b8154950 0x7fe8b814e4f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 --2- 172.21.15.123:0/2435327926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b80d1120 0x7fe8b81f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 >> 172.21.15.123:0/2435327926 conn(0x7fe8b80d1ae0 msgr2=0x7fe8b80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:25.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 shutdown_connections 2024-04-25T19:04:25.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.075+0000 7fe8bd800640 1 -- 172.21.15.123:0/2435327926 wait complete. 2024-04-25T19:04:25.102 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T19:04:25.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.356+0000 7fed79600640 1 Processor -- start 2024-04-25T19:04:25.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.356+0000 7fed79600640 1 -- start start 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed79600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed7415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed79600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed74162540 0x7fed740cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed79600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740cd380 con 0x7fed740ccf70 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed79600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fed7406deb0 con 0x7fed74162540 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed79600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed7406d610 con 0x7fed7406daa0 2024-04-25T19:04:25.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed73400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fed740ccf70 0x7fed740d0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36390/0 (socket says 172.21.15.123:36390) 2024-04-25T19:04:25.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.359+0000 7fed73400640 1 -- 172.21.15.123:0/2159814858 learned_addr learned my addr 172.21.15.123:0/2159814858 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:25.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed72000640 1 --2- 172.21.15.123:0/2159814858 >> v2:172.21.15.146:3300/0 conn(0x7fed74162540 0x7fed740cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed72a00640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed7415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016837882 0 0) 0x7fed6c001490 con 0x7fed740ccf70 2024-04-25T19:04:25.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed58003520 con 0x7fed740ccf70 2024-04-25T19:04:25.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3324731605 0 0) 0x7fed58003520 con 0x7fed740ccf70 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 msgr2=0x7fed7415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed7415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 >> v2:172.21.15.146:3300/0 conn(0x7fed74162540 msgr2=0x7fed740cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 --2- 172.21.15.123:0/2159814858 >> v2:172.21.15.146:3300/0 conn(0x7fed74162540 0x7fed740cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740cc5f0 con 0x7fed740ccf70 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fed6c001bc0 con 0x7fed740ccf70 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 >> v1:172.21.15.123:6789/0 conn(0x7fed740ccf70 legacy=0x7fed740d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.360+0000 7fed71600640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed580039b0 0x7fed58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed71600640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed58004800 0x7fed58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed71600640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed58005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed7406deb0 con 0x7fed58004800 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed7406d610 con 0x7fed580039b0 2024-04-25T19:04:25.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed5800b9c0 con 0x7fed7406daa0 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed58004800 0x7fed58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72a00640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed580039b0 0x7fed58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed73400640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed58005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 -- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 msgr2=0x7fed58005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed58005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 -- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed580039b0 msgr2=0x7fed58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed580039b0 0x7fed58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed72000640 1 -- 172.21.15.123:0/2159814858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed58003520 con 0x7fed58004800 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.361+0000 7fed73400640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7406daa0 0x7fed58005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.362+0000 7fed72000640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed58004800 0x7fed58004c10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed740cfde0 tx=0x7fed58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570b854f8e8d975d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60036070 con 0x7fed58004800 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60031400 con 0x7fed58004800 2024-04-25T19:04:25.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed71600640 1 -- 172.21.15.123:0/2159814858 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60009690 con 0x7fed58004800 2024-04-25T19:04:25.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed79600640 1 -- 172.21.15.123:0/2159814858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed58004800 msgr2=0x7fed58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed79600640 1 --2- 172.21.15.123:0/2159814858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed58004800 0x7fed58004c10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed740cfde0 tx=0x7fed58005830 comp rx=0 tx=0).stop 2024-04-25T19:04:25.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed73400640 1 -- 172.21.15.123:0/2159814858 reap_dead start 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed79600640 1 -- 172.21.15.123:0/2159814858 shutdown_connections 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.363+0000 7fed79600640 1 -- 172.21.15.123:0/2159814858 >> 172.21.15.123:0/2159814858 conn(0x7fed740ce390 msgr2=0x7fed740ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.364+0000 7fed79600640 1 -- 172.21.15.123:0/2159814858 shutdown_connections 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.364+0000 7fed79600640 1 -- 172.21.15.123:0/2159814858 wait complete. 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.366+0000 7fed79600640 1 Processor -- start 2024-04-25T19:04:25.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.366+0000 7fed79600640 1 -- start start 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed58004800 0x7fed74156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7409b850 0x7fed7409bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed7406cfc0 con 0x7fed740af000 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed7406cce0 con 0x7fed7409b850 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed79600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed7406d2a0 con 0x7fed58004800 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed73400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41038/0 (socket says 172.21.15.123:41038) 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed73400640 1 -- 172.21.15.123:0/2420381598 learned_addr learned my addr 172.21.15.123:0/2420381598 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:25.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed72000640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7409b850 0x7fed7409bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed72a00640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed58004800 0x7fed74156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed73400640 1 -- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed58004800 msgr2=0x7fed74156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.368+0000 7fed73400640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed58004800 0x7fed74156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed73400640 1 -- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7409b850 msgr2=0x7fed7409bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed73400640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7409b850 0x7fed7409bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed73400640 1 -- 172.21.15.123:0/2420381598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740af950 con 0x7fed740af000 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed73400640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed740cdb90 tx=0x7fed6c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00de70 con 0x7fed740af000 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c00bb80 con 0x7fed740af000 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c007070 con 0x7fed740af000 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7409df50 con 0x7fed740af000 2024-04-25T19:04:25.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.369+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7409e730 con 0x7fed740af000 2024-04-25T19:04:25.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.370+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7409e1a0 con 0x7fed740af000 2024-04-25T19:04:25.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.372+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed6c01b020 con 0x7fed740af000 2024-04-25T19:04:25.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.373+0000 7fed6aa00640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 0x7fed540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:25.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.373+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fed6c0956d0 con 0x7fed740af000 2024-04-25T19:04:25.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.373+0000 7fed72a00640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 0x7fed540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:25.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.373+0000 7fed72a00640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 0x7fed540769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fed64009160 tx=0x7fed64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:25.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.377+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed740af950 con 0x7fed740af000 2024-04-25T19:04:25.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:25.611+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fed74069c60 con 0x7fed740af000 2024-04-25T19:04:26.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.061+0000 7fed6aa00640 1 -- 172.21.15.123:0/2420381598 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fed740cd380 con 0x7fed740af000 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.064+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 msgr2=0x7fed540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.064+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 0x7fed540769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fed64009160 tx=0x7fed64008040 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.064+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 msgr2=0x7fed740af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.064+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed740cdb90 tx=0x7fed6c00b4a0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 shutdown_connections 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fed540744b0 0x7fed540769c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed740af000 0x7fed740af410 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7409b850 0x7fed7409bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 --2- 172.21.15.123:0/2420381598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed58004800 0x7fed74156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 >> 172.21.15.123:0/2420381598 conn(0x7fed740ce390 msgr2=0x7fed7406f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 shutdown_connections 2024-04-25T19:04:26.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.065+0000 7fed79600640 1 -- 172.21.15.123:0/2420381598 wait complete. 2024-04-25T19:04:26.100 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T19:04:26.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T19:04:26.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.376+0000 7fb048a00640 1 Processor -- start 2024-04-25T19:04:26.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.376+0000 7fb048a00640 1 -- start start 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb048a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb0440cfff0 0x7fb04415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb048a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb044143040 0x7fb04406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb048a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04415e2e0 con 0x7fb04406dac0 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb048a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb0440d1200 con 0x7fb0440cfff0 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb048a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb04406ded0 con 0x7fb044143040 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb043200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb04406dac0 0x7fb044160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36402/0 (socket says 172.21.15.123:36402) 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb043200640 1 -- 172.21.15.123:0/239275515 learned_addr learned my addr 172.21.15.123:0/239275515 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:26.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.378+0000 7fb042800640 1 --2- 172.21.15.123:0/239275515 >> v2:172.21.15.123:3300/0 conn(0x7fb0440cfff0 0x7fb04415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb041e00640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb044143040 0x7fb04406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb041400640 1 -- 172.21.15.123:0/239275515 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954832869 0 0) 0x7fb03c001110 con 0x7fb04406dac0 2024-04-25T19:04:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb041400640 1 -- 172.21.15.123:0/239275515 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb028003520 con 0x7fb04406dac0 2024-04-25T19:04:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb042800640 1 -- 172.21.15.123:0/239275515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb044143040 msgr2=0x7fb04406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:26.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb042800640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb044143040 0x7fb04406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb042800640 1 -- 172.21.15.123:0/239275515 >> v1:172.21.15.123:6789/0 conn(0x7fb04406dac0 legacy=0x7fb044160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.379+0000 7fb042800640 1 -- 172.21.15.123:0/239275515 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0440d05f0 con 0x7fb0440cfff0 2024-04-25T19:04:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.380+0000 7fb042800640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440cfff0 0x7fb04415dda0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4311ccb57ee79a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.380+0000 7fb041400640 1 -- 172.21.15.123:0/239275515 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034007740 con 0x7fb0440cfff0 2024-04-25T19:04:26.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.381+0000 7fb041400640 1 -- 172.21.15.123:0/239275515 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0340078e0 con 0x7fb0440cfff0 2024-04-25T19:04:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.381+0000 7fb041400640 1 -- 172.21.15.123:0/239275515 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034007c90 con 0x7fb0440cfff0 2024-04-25T19:04:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.381+0000 7fb048a00640 1 -- 172.21.15.123:0/239275515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440cfff0 msgr2=0x7fb04415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.381+0000 7fb048a00640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440cfff0 0x7fb04415dda0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03402a660 comp rx=0 tx=0).stop 2024-04-25T19:04:26.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 -- 172.21.15.123:0/239275515 shutdown_connections 2024-04-25T19:04:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb044143040 0x7fb04406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 --2- 172.21.15.123:0/239275515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440cfff0 0x7fb04415dda0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 -- 172.21.15.123:0/239275515 >> 172.21.15.123:0/239275515 conn(0x7fb044073090 msgr2=0x7fb0440734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 -- 172.21.15.123:0/239275515 shutdown_connections 2024-04-25T19:04:26.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.382+0000 7fb048a00640 1 -- 172.21.15.123:0/239275515 wait complete. 2024-04-25T19:04:26.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.384+0000 7fb048a00640 1 Processor -- start 2024-04-25T19:04:26.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.384+0000 7fb048a00640 1 -- start start 2024-04-25T19:04:26.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 0x7fb04413d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0440cfff0 0x7fb0441566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0440d0e90 con 0x7fb044143040 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb0440d0bb0 con 0x7fb04406dac0 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb048a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb044143450 con 0x7fb0440cfff0 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb043200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb043200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41068/0 (socket says 172.21.15.123:41068) 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.386+0000 7fb043200640 1 -- 172.21.15.123:0/201319857 learned_addr learned my addr 172.21.15.123:0/201319857 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb042800640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 0x7fb04413d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb041e00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0440cfff0 0x7fb0441566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 -- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0440cfff0 msgr2=0x7fb0441566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0440cfff0 0x7fb0441566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 -- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 msgr2=0x7fb04413d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 0x7fb04413d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 -- 172.21.15.123:0/201319857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04413d9e0 con 0x7fb044143040 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb042800640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 0x7fb04413d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:26.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb043200640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c3c0 tx=0x7fb03c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:26.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c017070 con 0x7fb044143040 2024-04-25T19:04:26.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c005ce0 con 0x7fb044143040 2024-04-25T19:04:26.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04413dcc0 con 0x7fb044143040 2024-04-25T19:04:26.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.387+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0072d0 con 0x7fb044143040 2024-04-25T19:04:26.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.388+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04413e280 con 0x7fb044143040 2024-04-25T19:04:26.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.389+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04413d750 con 0x7fb044143040 2024-04-25T19:04:26.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.390+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb03c01a070 con 0x7fb044143040 2024-04-25T19:04:26.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.391+0000 7fb03aa00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 0x7fb0200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:26.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.391+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb03c0945b0 con 0x7fb044143040 2024-04-25T19:04:26.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.391+0000 7fb042800640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 0x7fb0200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:26.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.392+0000 7fb042800640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 0x7fb0200769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb034034000 tx=0x7fb03402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:26.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.396+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb04415e2e0 con 0x7fb044143040 2024-04-25T19:04:26.421 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-25T19:04:26.420+0000 7ffac6200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T19:04:26.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:26.641+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb04413e660 con 0x7fb044143040 2024-04-25T19:04:27.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.074+0000 7fb03aa00640 1 -- 172.21.15.123:0/201319857 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fb03c05e5e0 con 0x7fb044143040 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 msgr2=0x7fb0200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 0x7fb0200769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb034034000 tx=0x7fb03402f000 comp rx=0 tx=0).stop 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 msgr2=0x7fb04415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c3c0 tx=0x7fb03c00c880 comp rx=0 tx=0).stop 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 shutdown_connections 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fb0200744b0 0x7fb0200769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb044143040 0x7fb04415bdf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0440cfff0 0x7fb0441566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.077+0000 7fb048a00640 1 --2- 172.21.15.123:0/201319857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb04406dac0 0x7fb04413d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.078+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 >> 172.21.15.123:0/201319857 conn(0x7fb044073090 msgr2=0x7fb0440703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:27.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.078+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 shutdown_connections 2024-04-25T19:04:27.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.078+0000 7fb048a00640 1 -- 172.21.15.123:0/201319857 wait complete. 2024-04-25T19:04:27.108 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T19:04:27.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:27.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.377+0000 7f9eaf200640 1 Processor -- start 2024-04-25T19:04:27.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.377+0000 7f9eaf200640 1 -- start start 2024-04-25T19:04:27.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eaf200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9eb0167b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eaf200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb00b5650 con 0x7f9eb013fbf0 2024-04-25T19:04:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eaf200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb00b5c70 con 0x7f9eb006dc90 2024-04-25T19:04:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eaf200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb0143050 con 0x7f9eb0142c40 2024-04-25T19:04:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eade00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9eb013fbf0 0x7f9eb006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36412/0 (socket says 172.21.15.123:36412) 2024-04-25T19:04:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eade00640 1 -- 172.21.15.123:0/3382524874 learned_addr learned my addr 172.21.15.123:0/3382524874 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9eae800640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9eb0167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.380+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001021383 0 0) 0x7f9eb00b5650 con 0x7f9eb013fbf0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003520 con 0x7f9eb013fbf0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4227461379 0 0) 0x7f9e94003520 con 0x7f9eb013fbf0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 msgr2=0x7f9eb0167b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9eb0167b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 >> v1:172.21.15.146:6789/0 conn(0x7f9eb006dc90 legacy=0x7f9eb0142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb00b5f50 con 0x7f9eb013fbf0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f9ea0001670 con 0x7f9eb013fbf0 2024-04-25T19:04:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 >> v1:172.21.15.123:6789/0 conn(0x7f9eb013fbf0 legacy=0x7f9eb006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e940039e0 0x7f9e94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e94004800 0x7f9e94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9e94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0143050 con 0x7f9e940039e0 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400baf0 con 0x7f9e94004800 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9400bcf0 con 0x7f9eb0142c40 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.381+0000 7f9eade00640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e940039e0 0x7f9e94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9ea7e00640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e94004800 0x7f9e94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eae800640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9e94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eade00640 1 -- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 msgr2=0x7f9e94005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eade00640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb0142c40 0x7f9e94005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eade00640 1 -- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e94004800 msgr2=0x7f9e94004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eade00640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e94004800 0x7f9e94004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.382+0000 7f9eade00640 1 -- 172.21.15.123:0/3382524874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94003520 con 0x7f9e940039e0 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.383+0000 7f9eade00640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e940039e0 0x7f9e94003e50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c210 tx=0x7f9e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c9a7c67126631d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.383+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007cf0 con 0x7f9e940039e0 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.383+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea0005af0 con 0x7f9e940039e0 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.383+0000 7f9ead400640 1 -- 172.21.15.123:0/3382524874 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007070 con 0x7f9e940039e0 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 -- 172.21.15.123:0/3382524874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e940039e0 msgr2=0x7f9e94003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 --2- 172.21.15.123:0/3382524874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e940039e0 0x7f9e94003e50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c210 tx=0x7f9e94005830 comp rx=0 tx=0).stop 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eae800640 1 -- 172.21.15.123:0/3382524874 reap_dead start 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 -- 172.21.15.123:0/3382524874 shutdown_connections 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 -- 172.21.15.123:0/3382524874 >> 172.21.15.123:0/3382524874 conn(0x7f9eb0073200 msgr2=0x7f9eb0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 -- 172.21.15.123:0/3382524874 shutdown_connections 2024-04-25T19:04:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.384+0000 7f9eaf200640 1 -- 172.21.15.123:0/3382524874 wait complete. 2024-04-25T19:04:27.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.387+0000 7f9eaf200640 1 Processor -- start 2024-04-25T19:04:27.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.387+0000 7f9eaf200640 1 -- start start 2024-04-25T19:04:27.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e940039e0 0x7f9eb015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb01393b0 0x7f9eb0152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00b6a30 con 0x7f9eb015ed80 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00b6510 con 0x7f9e940039e0 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eaf200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb006f130 con 0x7f9eb01393b0 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.388+0000 7f9eae800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41084/0 (socket says 172.21.15.123:41084) 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 -- 172.21.15.123:0/1340179 learned_addr learned my addr 172.21.15.123:0/1340179 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9ea7e00640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb01393b0 0x7f9eb0152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eade00640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e940039e0 0x7f9eb015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 -- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb01393b0 msgr2=0x7f9eb0152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb01393b0 0x7f9eb0152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 -- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e940039e0 msgr2=0x7f9eb015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e940039e0 0x7f9eb015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 -- 172.21.15.123:0/1340179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb015f6d0 con 0x7f9eb015ed80 2024-04-25T19:04:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9eae800640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00d14c0 tx=0x7f9ea800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.389+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea801c070 con 0x7f9eb015ed80 2024-04-25T19:04:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.390+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea8017400 con 0x7f9eb015ed80 2024-04-25T19:04:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.390+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80177f0 con 0x7f9eb015ed80 2024-04-25T19:04:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.390+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb0138310 con 0x7f9eb015ed80 2024-04-25T19:04:27.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.390+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01388d0 con 0x7f9eb015ed80 2024-04-25T19:04:27.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.391+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0138080 con 0x7f9eb015ed80 2024-04-25T19:04:27.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.392+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ea8005d10 con 0x7f9eb015ed80 2024-04-25T19:04:27.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.393+0000 7f9ea6000640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 0x7f9e8c076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.393+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9ea8095570 con 0x7f9eb015ed80 2024-04-25T19:04:27.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.393+0000 7f9eade00640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 0x7f9e8c076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.394+0000 7f9eade00640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 0x7f9e8c076990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0032000 tx=0x7f9ea0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.398+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9eb015f6d0 con 0x7f9eb015ed80 2024-04-25T19:04:27.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.641+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eb00b5890 con 0x7f9eb015ed80 2024-04-25T19:04:27.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.642+0000 7f9ea6000640 1 -- 172.21.15.123:0/1340179 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7f9ea8012070 con 0x7f9eb015ed80 2024-04-25T19:04:27.644 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:27.644 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:04:27.644 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.644+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 msgr2=0x7f9e8c076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.644+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 0x7f9e8c076990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0032000 tx=0x7f9ea0009030 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.644+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 msgr2=0x7f9eb015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.644+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00d14c0 tx=0x7f9ea800f4a0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 shutdown_connections 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f9e8c074480 0x7f9e8c076990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eb015ed80 0x7f9eb015f190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9eb01393b0 0x7f9eb0152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 --2- 172.21.15.123:0/1340179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e940039e0 0x7f9eb015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 >> 172.21.15.123:0/1340179 conn(0x7f9eb0073200 msgr2=0x7f9eb006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 shutdown_connections 2024-04-25T19:04:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.645+0000 7f9eaf200640 1 -- 172.21.15.123:0/1340179 wait complete. 2024-04-25T19:04:27.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:04:27.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.930+0000 7f65efe00640 1 Processor -- start 2024-04-25T19:04:27.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.930+0000 7f65efe00640 1 -- start start 2024-04-25T19:04:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.932+0000 7f65efe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.932+0000 7f65efe00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 0x7f65f0142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.932+0000 7f65efe00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f006ded0 con 0x7f65f0142a70 2024-04-25T19:04:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.932+0000 7f65efe00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f65f00b5650 con 0x7f65f006dac0 2024-04-25T19:04:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.932+0000 7f65efe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f65f00b5c70 con 0x7f65f013fbf0 2024-04-25T19:04:27.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.933+0000 7f65ee000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 0x7f65f0142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.933+0000 7f65eea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.933+0000 7f65ee000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 0x7f65f0142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.123:34466/0 (socket says 172.21.15.123:34466) 2024-04-25T19:04:27.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.933+0000 7f65ee000640 1 -- 172.21.15.123:0/2662466374 learned_addr learned my addr 172.21.15.123:0/2662466374 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.933+0000 7f65eea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:50296/0 (socket says 172.21.15.123:50296) 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.934+0000 7f65eea00640 1 -- 172.21.15.123:0/2662466374 >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 msgr2=0x7f65f0142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.934+0000 7f65eea00640 1 --2- 172.21.15.123:0/2662466374 >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 0x7f65f0142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.934+0000 7f65eea00640 1 -- 172.21.15.123:0/2662466374 >> v1:172.21.15.146:6789/0 conn(0x7f65f0142a70 legacy=0x7f65f01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.934+0000 7f65eea00640 1 -- 172.21.15.123:0/2662466374 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f00b5f50 con 0x7f65f013fbf0 2024-04-25T19:04:27.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.935+0000 7f65eea00640 1 --2- 172.21.15.123:0/2662466374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65e00097a0 tx=0x7f65e002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46594dcb88a0ee15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.935+0000 7f65ed600640 1 -- 172.21.15.123:0/2662466374 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0037070 con 0x7f65f013fbf0 2024-04-25T19:04:27.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.935+0000 7f65ed600640 1 -- 172.21.15.123:0/2662466374 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e0032440 con 0x7f65f013fbf0 2024-04-25T19:04:27.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.935+0000 7f65ed600640 1 -- 172.21.15.123:0/2662466374 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0032810 con 0x7f65f013fbf0 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 -- 172.21.15.123:0/2662466374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 msgr2=0x7f65f006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 --2- 172.21.15.123:0/2662466374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65e00097a0 tx=0x7f65e002a180 comp rx=0 tx=0).stop 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 -- 172.21.15.123:0/2662466374 shutdown_connections 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 --2- 172.21.15.123:0/2662466374 >> v2:172.21.15.146:3300/0 conn(0x7f65f006dac0 0x7f65f0142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 --2- 172.21.15.123:0/2662466374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f013fbf0 0x7f65f006d580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 -- 172.21.15.123:0/2662466374 >> 172.21.15.123:0/2662466374 conn(0x7f65f0073200 msgr2=0x7f65f0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 -- 172.21.15.123:0/2662466374 shutdown_connections 2024-04-25T19:04:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.936+0000 7f65efe00640 1 -- 172.21.15.123:0/2662466374 wait complete. 2024-04-25T19:04:27.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.938+0000 7f65efe00640 1 Processor -- start 2024-04-25T19:04:27.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.939+0000 7f65efe00640 1 -- start start 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 0x7f65f0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65f013fbf0 0x7f65f00ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00b6a30 con 0x7f65f0142a70 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00b6510 con 0x7f65f013fbf0 2024-04-25T19:04:27.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65efe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f65f0142e80 con 0x7f65f006dac0 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41098/0 (socket says 172.21.15.123:41098) 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.940+0000 7f65ef400640 1 -- 172.21.15.123:0/579877894 learned_addr learned my addr 172.21.15.123:0/579877894 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ee000640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65f013fbf0 0x7f65f00ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65eea00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 0x7f65f0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 -- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 msgr2=0x7f65f0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 0x7f65f0153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 -- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65f013fbf0 msgr2=0x7f65f00ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65f013fbf0 0x7f65f00ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 -- 172.21.15.123:0/579877894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f01546f0 con 0x7f65f0142a70 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65eea00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 0x7f65f0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65ef400640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65e800c3f0 tx=0x7f65e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8017070 con 0x7f65f0142a70 2024-04-25T19:04:27.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e80052d0 con 0x7f65f0142a70 2024-04-25T19:04:27.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f01549d0 con 0x7f65f0142a70 2024-04-25T19:04:27.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.941+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8003830 con 0x7f65f0142a70 2024-04-25T19:04:27.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.942+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f0154f90 con 0x7f65f0142a70 2024-04-25T19:04:27.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.943+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f0050740 con 0x7f65f0142a70 2024-04-25T19:04:27.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.944+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f65e801a070 con 0x7f65f0142a70 2024-04-25T19:04:27.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.945+0000 7f65e7400640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 0x7f65cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:27.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.945+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f65e8094750 con 0x7f65f0142a70 2024-04-25T19:04:27.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.945+0000 7f65eea00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 0x7f65cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:27.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.946+0000 7f65eea00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 0x7f65cc0769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65e0034000 tx=0x7f65e002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:27.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:27.950+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65f006ded0 con 0x7f65f0142a70 2024-04-25T19:04:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.151+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65f01f6680 con 0x7f65f0142a70 2024-04-25T19:04:28.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.155+0000 7f65e7400640 1 -- 172.21.15.123:0/579877894 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16990 (secure 0 0 0) 0x7f65e805ed70 con 0x7f65f0142a70 2024-04-25T19:04:28.157 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:28.157 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":40,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:04:25.027549+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 msgr2=0x7f65cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 0x7f65cc0769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65e0034000 tx=0x7f65e002e000 comp rx=0 tx=0).stop 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 msgr2=0x7f65f00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65e800c3f0 tx=0x7f65e800c7f0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 shutdown_connections 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65cc0744b0 0x7f65cc0769c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f0142a70 0x7f65f00cf2d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65f013fbf0 0x7f65f00ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 --2- 172.21.15.123:0/579877894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65f006dac0 0x7f65f0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 >> 172.21.15.123:0/579877894 conn(0x7f65f0073200 msgr2=0x7f65f00b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:28.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 shutdown_connections 2024-04-25T19:04:28.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.157+0000 7f65efe00640 1 -- 172.21.15.123:0/579877894 wait complete. 2024-04-25T19:04:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T19:04:28.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:04:28.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.434+0000 7f9456800640 1 Processor -- start 2024-04-25T19:04:28.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.434+0000 7f9456800640 1 -- start start 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.436+0000 7f9456800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.436+0000 7f9456800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9450142c40 0x7f94500b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.436+0000 7f9456800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945006ded0 con 0x7f945006dac0 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.436+0000 7f9456800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9450143050 con 0x7f9450142c40 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.436+0000 7f9456800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f94500b5c70 con 0x7f945013fbf0 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9450142c40 0x7f94500b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9450142c40 0x7f94500b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41114/0 (socket says 172.21.15.123:41114) 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 -- 172.21.15.123:0/1903669684 learned_addr learned my addr 172.21.15.123:0/1903669684 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455400640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 -- 172.21.15.123:0/1903669684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 msgr2=0x7f945006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 -- 172.21.15.123:0/1903669684 >> v1:172.21.15.146:6789/0 conn(0x7f945006dac0 legacy=0x7f9450142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455e00640 1 -- 172.21.15.123:0/1903669684 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94500b5f50 con 0x7f9450142c40 2024-04-25T19:04:28.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.437+0000 7f9455400640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:28.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.438+0000 7f9455e00640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9450142c40 0x7f94500b4640 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9448009e50 tx=0x7f944802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4c57a12afe1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:28.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.439+0000 7f944fe00640 1 -- 172.21.15.123:0/1903669684 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448036070 con 0x7f9450142c40 2024-04-25T19:04:28.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.439+0000 7f944fe00640 1 -- 172.21.15.123:0/1903669684 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9448031600 con 0x7f9450142c40 2024-04-25T19:04:28.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.439+0000 7f944fe00640 1 -- 172.21.15.123:0/1903669684 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94480319d0 con 0x7f9450142c40 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 -- 172.21.15.123:0/1903669684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9450142c40 msgr2=0x7f94500b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9450142c40 0x7f94500b4640 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9448009e50 tx=0x7f944802a6d0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 -- 172.21.15.123:0/1903669684 shutdown_connections 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9450142c40 0x7f94500b4640 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 --2- 172.21.15.123:0/1903669684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 -- 172.21.15.123:0/1903669684 >> 172.21.15.123:0/1903669684 conn(0x7f9450073200 msgr2=0x7f9450073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 -- 172.21.15.123:0/1903669684 shutdown_connections 2024-04-25T19:04:28.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.440+0000 7f9456800640 1 -- 172.21.15.123:0/1903669684 wait complete. 2024-04-25T19:04:28.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.442+0000 7f9456800640 1 Processor -- start 2024-04-25T19:04:28.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.442+0000 7f9456800640 1 -- start start 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9450142c40 0x7f94500cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94500b6a30 con 0x7f945006dac0 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f94500b6510 con 0x7f9450142c40 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9456800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f94500b4b80 con 0x7f945013fbf0 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41126/0 (socket says 172.21.15.123:41126) 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 -- 172.21.15.123:0/2906109271 learned_addr learned my addr 172.21.15.123:0/2906109271 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 -- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 msgr2=0x7f945014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 -- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9450142c40 msgr2=0x7f94500cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9450142c40 0x7f94500cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.444+0000 7f9455400640 1 -- 172.21.15.123:0/2906109271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945014ea40 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f9455400640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f944400c820 tx=0x7f944400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444015070 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9444007a30 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444007de0 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945013d740 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.445+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9450154370 con 0x7f945006dac0 2024-04-25T19:04:28.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.446+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9450161680 con 0x7f945006dac0 2024-04-25T19:04:28.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.447+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9444018070 con 0x7f945006dac0 2024-04-25T19:04:28.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.448+0000 7f944d600640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 0x7f94300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.448+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9444095360 con 0x7f945006dac0 2024-04-25T19:04:28.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.448+0000 7f9454a00640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 0x7f94300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.449+0000 7f9454a00640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 0x7f94300769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:28.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.453+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9450161680 con 0x7f945006dac0 2024-04-25T19:04:28.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.633+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9450050740 con 0x7f94300744b0 2024-04-25T19:04:28.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.643+0000 7f944d600640 1 -- 172.21.15.123:0/2906109271 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241542 (secure 0 0 0) 0x7f9450050740 con 0x7f94300744b0 2024-04-25T19:04:28.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:28.649 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 msgr2=0x7f94300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 0x7f94300769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c006bc0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 msgr2=0x7f9450138e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f944400c820 tx=0x7f944400cce0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 shutdown_connections 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f94300744b0 0x7f94300769c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9450142c40 0x7f94500cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f945013fbf0 0x7f945014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 --2- 172.21.15.123:0/2906109271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f945006dac0 0x7f9450138e60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 >> 172.21.15.123:0/2906109271 conn(0x7f9450073200 msgr2=0x7f945015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 shutdown_connections 2024-04-25T19:04:28.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.649+0000 7f9456800640 1 -- 172.21.15.123:0/2906109271 wait complete. 2024-04-25T19:04:28.675 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-04-25T19:04:27.002491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220604,"kb_used_data":4380,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754116,"statfs":{"total":773094113280,"available":772868214784,"internally_reserved":0,"allocated":4485120,"data_stored":2549641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.012775"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042886+0000","last_change":"2024-04-25T19:04:24.032304+0000","last_active":"2024-04-25T19:04:25.042886+0000","last_peered":"2024-04-25T19:04:25.042886+0000","last_clean":"2024-04-25T19:04:25.042886+0000","last_became_active":"2024-04-25T19:04:24.031169+0000","last_became_peered":"2024-04-25T19:04:24.031169+0000","last_unstale":"2024-04-25T19:04:25.042886+0000","last_undegraded":"2024-04-25T19:04:25.042886+0000","last_fullsized":"2024-04-25T19:04:25.042886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:35:47.211518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040396+0000","last_change":"2024-04-25T19:04:24.027966+0000","last_active":"2024-04-25T19:04:25.040396+0000","last_peered":"2024-04-25T19:04:25.040396+0000","last_clean":"2024-04-25T19:04:25.040396+0000","last_became_active":"2024-04-25T19:04:24.026879+0000","last_became_peered":"2024-04-25T19:04:24.026879+0000","last_unstale":"2024-04-25T19:04:25.040396+0000","last_undegraded":"2024-04-25T19:04:25.040396+0000","last_fullsized":"2024-04-25T19:04:25.040396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:55.866046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.009993+0000","last_change":"2024-04-25T19:04:22.009993+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.009993+0000","last_undegraded":"2024-04-25T19:04:22.009993+0000","last_fullsized":"2024-04-25T19:04:22.009993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041281+0000","last_change":"2024-04-25T19:04:24.040734+0000","last_active":"2024-04-25T19:04:25.041281+0000","last_peered":"2024-04-25T19:04:25.041281+0000","last_clean":"2024-04-25T19:04:25.041281+0000","last_became_active":"2024-04-25T19:04:24.040397+0000","last_became_peered":"2024-04-25T19:04:24.040397+0000","last_unstale":"2024-04-25T19:04:25.041281+0000","last_undegraded":"2024-04-25T19:04:25.041281+0000","last_fullsized":"2024-04-25T19:04:25.041281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:17.487495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043204+0000","last_change":"2024-04-25T19:04:23.035885+0000","last_active":"2024-04-25T19:04:25.043204+0000","last_peered":"2024-04-25T19:04:25.043204+0000","last_clean":"2024-04-25T19:04:25.043204+0000","last_became_active":"2024-04-25T19:04:23.035094+0000","last_became_peered":"2024-04-25T19:04:23.035094+0000","last_unstale":"2024-04-25T19:04:25.043204+0000","last_undegraded":"2024-04-25T19:04:25.043204+0000","last_fullsized":"2024-04-25T19:04:25.043204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:29.579519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:21.997704+0000","last_change":"2024-04-25T19:04:21.997704+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:21.997704+0000","last_undegraded":"2024-04-25T19:04:21.997704+0000","last_fullsized":"2024-04-25T19:04:21.997704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042085+0000","last_change":"2024-04-25T19:04:24.043120+0000","last_active":"2024-04-25T19:04:25.042085+0000","last_peered":"2024-04-25T19:04:25.042085+0000","last_clean":"2024-04-25T19:04:25.042085+0000","last_became_active":"2024-04-25T19:04:24.042756+0000","last_became_peered":"2024-04-25T19:04:24.042756+0000","last_unstale":"2024-04-25T19:04:25.042085+0000","last_undegraded":"2024-04-25T19:04:25.042085+0000","last_fullsized":"2024-04-25T19:04:25.042085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:17:35.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.010384+0000","last_change":"2024-04-25T19:04:22.010384+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.010384+0000","last_undegraded":"2024-04-25T19:04:22.010384+0000","last_fullsized":"2024-04-25T19:04:22.010384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.011032+0000","last_change":"2024-04-25T19:04:22.011032+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.011032+0000","last_undegraded":"2024-04-25T19:04:22.011032+0000","last_fullsized":"2024-04-25T19:04:22.011032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.002485+0000","last_change":"2024-04-25T19:04:22.002485+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.002485+0000","last_undegraded":"2024-04-25T19:04:22.002485+0000","last_fullsized":"2024-04-25T19:04:22.002485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:21.997020+0000","last_change":"2024-04-25T19:04:21.997020+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:21.997020+0000","last_undegraded":"2024-04-25T19:04:21.997020+0000","last_fullsized":"2024-04-25T19:04:21.997020+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040344+0000","last_change":"2024-04-25T19:04:24.035929+0000","last_active":"2024-04-25T19:04:25.040344+0000","last_peered":"2024-04-25T19:04:25.040344+0000","last_clean":"2024-04-25T19:04:25.040344+0000","last_became_active":"2024-04-25T19:04:24.035598+0000","last_became_peered":"2024-04-25T19:04:24.035598+0000","last_unstale":"2024-04-25T19:04:25.040344+0000","last_undegraded":"2024-04-25T19:04:25.040344+0000","last_fullsized":"2024-04-25T19:04:25.040344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:16.266368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.005391+0000","last_change":"2024-04-25T19:04:22.005391+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.005391+0000","last_undegraded":"2024-04-25T19:04:22.005391+0000","last_fullsized":"2024-04-25T19:04:22.005391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.000861+0000","last_change":"2024-04-25T19:04:22.000861+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.000861+0000","last_undegraded":"2024-04-25T19:04:22.000861+0000","last_fullsized":"2024-04-25T19:04:22.000861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.002631+0000","last_change":"2024-04-25T19:04:22.002631+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.002631+0000","last_undegraded":"2024-04-25T19:04:22.002631+0000","last_fullsized":"2024-04-25T19:04:22.002631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.001078+0000","last_change":"2024-04-25T19:04:22.001078+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.001078+0000","last_undegraded":"2024-04-25T19:04:22.001078+0000","last_fullsized":"2024-04-25T19:04:22.001078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.005930+0000","last_change":"2024-04-25T19:04:22.005930+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.005930+0000","last_undegraded":"2024-04-25T19:04:22.005930+0000","last_fullsized":"2024-04-25T19:04:22.005930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.005686+0000","last_change":"2024-04-25T19:04:22.005686+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.005686+0000","last_undegraded":"2024-04-25T19:04:22.005686+0000","last_fullsized":"2024-04-25T19:04:22.005686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041078+0000","last_change":"2024-04-25T19:04:24.042094+0000","last_active":"2024-04-25T19:04:25.041078+0000","last_peered":"2024-04-25T19:04:25.041078+0000","last_clean":"2024-04-25T19:04:25.041078+0000","last_became_active":"2024-04-25T19:04:24.041343+0000","last_became_peered":"2024-04-25T19:04:24.041343+0000","last_unstale":"2024-04-25T19:04:25.041078+0000","last_undegraded":"2024-04-25T19:04:25.041078+0000","last_fullsized":"2024-04-25T19:04:25.041078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:41.267557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040771+0000","last_change":"2024-04-25T19:04:23.039863+0000","last_active":"2024-04-25T19:04:25.040771+0000","last_peered":"2024-04-25T19:04:25.040771+0000","last_clean":"2024-04-25T19:04:25.040771+0000","last_became_active":"2024-04-25T19:04:23.039505+0000","last_became_peered":"2024-04-25T19:04:23.039505+0000","last_unstale":"2024-04-25T19:04:25.040771+0000","last_undegraded":"2024-04-25T19:04:25.040771+0000","last_fullsized":"2024-04-25T19:04:25.040771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.507979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.002289+0000","last_change":"2024-04-25T19:04:22.002289+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.002289+0000","last_undegraded":"2024-04-25T19:04:22.002289+0000","last_fullsized":"2024-04-25T19:04:22.002289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.010167+0000","last_change":"2024-04-25T19:04:22.010167+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.010167+0000","last_undegraded":"2024-04-25T19:04:22.010167+0000","last_fullsized":"2024-04-25T19:04:22.010167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040951+0000","last_change":"2024-04-25T19:04:24.037283+0000","last_active":"2024-04-25T19:04:25.040951+0000","last_peered":"2024-04-25T19:04:25.040951+0000","last_clean":"2024-04-25T19:04:25.040951+0000","last_became_active":"2024-04-25T19:04:24.036962+0000","last_became_peered":"2024-04-25T19:04:24.036962+0000","last_unstale":"2024-04-25T19:04:25.040951+0000","last_undegraded":"2024-04-25T19:04:25.040951+0000","last_fullsized":"2024-04-25T19:04:25.040951+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:50.762653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.003309+0000","last_change":"2024-04-25T19:04:22.003309+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.003309+0000","last_undegraded":"2024-04-25T19:04:22.003309+0000","last_fullsized":"2024-04-25T19:04:22.003309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.001946+0000","last_change":"2024-04-25T19:04:22.001946+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.001946+0000","last_undegraded":"2024-04-25T19:04:22.001946+0000","last_fullsized":"2024-04-25T19:04:22.001946+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.001910+0000","last_change":"2024-04-25T19:04:22.001910+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.001910+0000","last_undegraded":"2024-04-25T19:04:22.001910+0000","last_fullsized":"2024-04-25T19:04:22.001910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.011206+0000","last_change":"2024-04-25T19:04:22.011206+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.011206+0000","last_undegraded":"2024-04-25T19:04:22.011206+0000","last_fullsized":"2024-04-25T19:04:22.011206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.011686+0000","last_change":"2024-04-25T19:04:22.011686+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.011686+0000","last_undegraded":"2024-04-25T19:04:22.011686+0000","last_fullsized":"2024-04-25T19:04:22.011686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042255+0000","last_change":"2024-04-25T19:04:23.032920+0000","last_active":"2024-04-25T19:04:25.042255+0000","last_peered":"2024-04-25T19:04:25.042255+0000","last_clean":"2024-04-25T19:04:25.042255+0000","last_became_active":"2024-04-25T19:04:23.032583+0000","last_became_peered":"2024-04-25T19:04:23.032583+0000","last_unstale":"2024-04-25T19:04:25.042255+0000","last_undegraded":"2024-04-25T19:04:25.042255+0000","last_fullsized":"2024-04-25T19:04:25.042255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:49.943413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.003805+0000","last_change":"2024-04-25T19:04:22.003805+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.003805+0000","last_undegraded":"2024-04-25T19:04:22.003805+0000","last_fullsized":"2024-04-25T19:04:22.003805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039250+0000","last_change":"2024-04-25T19:04:23.039339+0000","last_active":"2024-04-25T19:04:25.039250+0000","last_peered":"2024-04-25T19:04:25.039250+0000","last_clean":"2024-04-25T19:04:25.039250+0000","last_became_active":"2024-04-25T19:04:23.038657+0000","last_became_peered":"2024-04-25T19:04:23.038657+0000","last_unstale":"2024-04-25T19:04:25.039250+0000","last_undegraded":"2024-04-25T19:04:25.039250+0000","last_fullsized":"2024-04-25T19:04:25.039250+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:11.204701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.001674+0000","last_change":"2024-04-25T19:04:22.001674+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.001674+0000","last_undegraded":"2024-04-25T19:04:22.001674+0000","last_fullsized":"2024-04-25T19:04:22.001674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:21.999892+0000","last_change":"2024-04-25T19:04:21.999892+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:21.999892+0000","last_undegraded":"2024-04-25T19:04:21.999892+0000","last_fullsized":"2024-04-25T19:04:21.999892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.000622+0000","last_change":"2024-04-25T19:04:22.000622+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.000622+0000","last_undegraded":"2024-04-25T19:04:22.000622+0000","last_fullsized":"2024-04-25T19:04:22.000622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.004215+0000","last_change":"2024-04-25T19:04:22.004215+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.004215+0000","last_undegraded":"2024-04-25T19:04:22.004215+0000","last_fullsized":"2024-04-25T19:04:22.004215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041625+0000","last_change":"2024-04-25T19:04:24.043192+0000","last_active":"2024-04-25T19:04:25.041625+0000","last_peered":"2024-04-25T19:04:25.041625+0000","last_clean":"2024-04-25T19:04:25.041625+0000","last_became_active":"2024-04-25T19:04:24.042766+0000","last_became_peered":"2024-04-25T19:04:24.042766+0000","last_unstale":"2024-04-25T19:04:25.041625+0000","last_undegraded":"2024-04-25T19:04:25.041625+0000","last_fullsized":"2024-04-25T19:04:25.041625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:07.475303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.005928+0000","last_change":"2024-04-25T19:04:22.005928+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.005928+0000","last_undegraded":"2024-04-25T19:04:22.005928+0000","last_fullsized":"2024-04-25T19:04:22.005928+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.001038+0000","last_change":"2024-04-25T19:04:22.001038+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.001038+0000","last_undegraded":"2024-04-25T19:04:22.001038+0000","last_fullsized":"2024-04-25T19:04:22.001038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.002494+0000","last_change":"2024-04-25T19:04:22.002494+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.002494+0000","last_undegraded":"2024-04-25T19:04:22.002494+0000","last_fullsized":"2024-04-25T19:04:22.002494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T19:04:22.987829+0000","last_change":"2024-04-25T19:04:22.987829+0000","last_active":"2024-04-25T19:04:22.987829+0000","last_peered":"2024-04-25T19:04:22.987829+0000","last_clean":"2024-04-25T19:04:22.987829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.987829+0000","last_undegraded":"2024-04-25T19:04:22.987829+0000","last_fullsized":"2024-04-25T19:04:22.987829+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T19:04:22.006495+0000","last_change":"2024-04-25T19:04:22.006495+0000","last_active":"2024-04-25T19:04:21.972633+0000","last_peered":"2024-04-25T19:04:21.972633+0000","last_clean":"2024-04-25T19:04:21.972633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T19:04:22.006495+0000","last_undegraded":"2024-04-25T19:04:22.006495+0000","last_fullsized":"2024-04-25T19:04:22.006495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":137,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T19:04:22.006841+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:04:22.006841+0000","last_peered":"2024-04-25T19:04:22.006841+0000","last_clean":"2024-04-25T19:04:22.006841+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:04:22.006841+0000","last_undegraded":"2024-04-25T19:04:22.006841+0000","last_fullsized":"2024-04-25T19:04:22.006841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":758135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":999424,"data_stored":758135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:04:28.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:04:28.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.979+0000 7f2291400640 1 Processor -- start 2024-04-25T19:04:28.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.979+0000 7f2291400640 1 -- start start 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.981+0000 7f2291400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f228c06d580 0x7f228c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.981+0000 7f2291400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228c06ded0 con 0x7f228c06fe20 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.981+0000 7f2291400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228c15f070 con 0x7f228c15e2f0 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.981+0000 7f2291400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f228c15ee10 con 0x7f228c06d580 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f228c06d580 0x7f228c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f228c06fe20 0x7f228c15b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36416/0 (socket says 172.21.15.123:36416) 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228be00640 1 -- 172.21.15.123:0/448713890 learned_addr learned my addr 172.21.15.123:0/448713890 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228aa00640 1 -- 172.21.15.123:0/448713890 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490486588 0 0) 0x7f227c001110 con 0x7f228c06fe20 2024-04-25T19:04:28.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228aa00640 1 -- 172.21.15.123:0/448713890 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2270003520 con 0x7f228c06fe20 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228b400640 1 -- 172.21.15.123:0/448713890 >> v1:172.21.15.146:6789/0 conn(0x7f228c15e2f0 legacy=0x7f228c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228b400640 1 -- 172.21.15.123:0/448713890 >> v1:172.21.15.123:6789/0 conn(0x7f228c06fe20 legacy=0x7f228c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.982+0000 7f228b400640 1 -- 172.21.15.123:0/448713890 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c06f340 con 0x7f228c06d580 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.983+0000 7f228b400640 1 --2- 172.21.15.123:0/448713890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06d580 0x7f228c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f22780097a0 tx=0x7f227802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cabc8a0b152c09a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.984+0000 7f228aa00640 1 -- 172.21.15.123:0/448713890 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278037070 con 0x7f228c06d580 2024-04-25T19:04:28.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.984+0000 7f228aa00640 1 -- 172.21.15.123:0/448713890 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227802ae50 con 0x7f228c06d580 2024-04-25T19:04:28.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.984+0000 7f228aa00640 1 -- 172.21.15.123:0/448713890 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278032660 con 0x7f228c06d580 2024-04-25T19:04:28.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.984+0000 7f2291400640 1 -- 172.21.15.123:0/448713890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06d580 msgr2=0x7f228c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.984+0000 7f2291400640 1 --2- 172.21.15.123:0/448713890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06d580 0x7f228c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f22780097a0 tx=0x7f227802a140 comp rx=0 tx=0).stop 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.985+0000 7f2291400640 1 -- 172.21.15.123:0/448713890 shutdown_connections 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.985+0000 7f2291400640 1 --2- 172.21.15.123:0/448713890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06d580 0x7f228c06d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.985+0000 7f2291400640 1 -- 172.21.15.123:0/448713890 >> 172.21.15.123:0/448713890 conn(0x7f228c073200 msgr2=0x7f228c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.985+0000 7f2291400640 1 -- 172.21.15.123:0/448713890 shutdown_connections 2024-04-25T19:04:28.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.985+0000 7f2291400640 1 -- 172.21.15.123:0/448713890 wait complete. 2024-04-25T19:04:28.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.987+0000 7f2291400640 1 Processor -- start 2024-04-25T19:04:28.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.987+0000 7f2291400640 1 -- start start 2024-04-25T19:04:28.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f228c15e2f0 0x7f228c1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228c15c890 0x7f228c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f228c168c70 con 0x7f228c06fe20 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f228c06f900 con 0x7f228c15c890 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2291400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f228c168df0 con 0x7f228c15e2f0 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41136/0 (socket says 172.21.15.123:41136) 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 -- 172.21.15.123:0/4017476925 learned_addr learned my addr 172.21.15.123:0/4017476925 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228b400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f228c15e2f0 0x7f228c1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f2290a00640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228c15c890 0x7f228c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 -- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f228c15e2f0 msgr2=0x7f228c1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f228c15e2f0 0x7f228c1f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.989+0000 7f228be00640 1 -- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228c15c890 msgr2=0x7f228c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:28.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f228be00640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228c15c890 0x7f228c0cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:28.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f228be00640 1 -- 172.21.15.123:0/4017476925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c1f7a70 con 0x7f228c06fe20 2024-04-25T19:04:28.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f228be00640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c340 tx=0x7f227c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:28.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c017070 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c005ce0 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c13ad10 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c0072f0 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.990+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c13b290 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.991+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c15aef0 con 0x7f228c06fe20 2024-04-25T19:04:28.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.993+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f227c01a070 con 0x7f228c06fe20 2024-04-25T19:04:28.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.993+0000 7f2283e00640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 0x7f226c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:28.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.994+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f227c0947d0 con 0x7f228c06fe20 2024-04-25T19:04:28.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.994+0000 7f228b400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 0x7f226c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:28.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.994+0000 7f228b400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 0x7f226c0769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f227802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:28.999+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f228c1f7a70 con 0x7f228c06fe20 2024-04-25T19:04:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.196+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f228c070a00 con 0x7f226c0744b0 2024-04-25T19:04:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.205+0000 7f2283e00640 1 -- 172.21.15.123:0/4017476925 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f228c070a00 con 0x7f226c0744b0 2024-04-25T19:04:29.210 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 msgr2=0x7f226c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 0x7f226c0769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f227802e000 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 msgr2=0x7f228c1f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c340 tx=0x7f227c00c800 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 shutdown_connections 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f226c0744b0 0x7f226c0769c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228c15c890 0x7f228c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f228c15e2f0 0x7f228c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 --2- 172.21.15.123:0/4017476925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f228c06fe20 0x7f228c1f6440 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 >> 172.21.15.123:0/4017476925 conn(0x7f228c073200 msgr2=0x7f228c15fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.211+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 shutdown_connections 2024-04-25T19:04:29.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.212+0000 7f2291400640 1 -- 172.21.15.123:0/4017476925 wait complete. 2024-04-25T19:04:29.237 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T19:04:29.003151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220968,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753752,"statfs":{"total":773094113280,"available":772867842048,"internally_reserved":0,"allocated":4890624,"data_stored":2741572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004829"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041874+0000","last_change":"2024-04-25T19:04:24.027127+0000","last_active":"2024-04-25T19:04:25.041874+0000","last_peered":"2024-04-25T19:04:25.041874+0000","last_clean":"2024-04-25T19:04:25.041874+0000","last_became_active":"2024-04-25T19:04:24.026803+0000","last_became_peered":"2024-04-25T19:04:24.026803+0000","last_unstale":"2024-04-25T19:04:25.041874+0000","last_undegraded":"2024-04-25T19:04:25.041874+0000","last_fullsized":"2024-04-25T19:04:25.041874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:24.613090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042886+0000","last_change":"2024-04-25T19:04:24.032304+0000","last_active":"2024-04-25T19:04:25.042886+0000","last_peered":"2024-04-25T19:04:25.042886+0000","last_clean":"2024-04-25T19:04:25.042886+0000","last_became_active":"2024-04-25T19:04:24.031169+0000","last_became_peered":"2024-04-25T19:04:24.031169+0000","last_unstale":"2024-04-25T19:04:25.042886+0000","last_undegraded":"2024-04-25T19:04:25.042886+0000","last_fullsized":"2024-04-25T19:04:25.042886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:35:47.211518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042684+0000","last_change":"2024-04-25T19:04:24.043821+0000","last_active":"2024-04-25T19:04:25.042684+0000","last_peered":"2024-04-25T19:04:25.042684+0000","last_clean":"2024-04-25T19:04:25.042684+0000","last_became_active":"2024-04-25T19:04:24.042432+0000","last_became_peered":"2024-04-25T19:04:24.042432+0000","last_unstale":"2024-04-25T19:04:25.042684+0000","last_undegraded":"2024-04-25T19:04:25.042684+0000","last_fullsized":"2024-04-25T19:04:25.042684+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:42.660532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042499+0000","last_change":"2024-04-25T19:04:24.053554+0000","last_active":"2024-04-25T19:04:25.042499+0000","last_peered":"2024-04-25T19:04:25.042499+0000","last_clean":"2024-04-25T19:04:25.042499+0000","last_became_active":"2024-04-25T19:04:24.053144+0000","last_became_peered":"2024-04-25T19:04:24.053144+0000","last_unstale":"2024-04-25T19:04:25.042499+0000","last_undegraded":"2024-04-25T19:04:25.042499+0000","last_fullsized":"2024-04-25T19:04:25.042499+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:45.262293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070256+0000","last_change":"2024-04-25T19:04:24.039855+0000","last_active":"2024-04-25T19:04:25.070256+0000","last_peered":"2024-04-25T19:04:25.070256+0000","last_clean":"2024-04-25T19:04:25.070256+0000","last_became_active":"2024-04-25T19:04:24.039510+0000","last_became_peered":"2024-04-25T19:04:24.039510+0000","last_unstale":"2024-04-25T19:04:25.070256+0000","last_undegraded":"2024-04-25T19:04:25.070256+0000","last_fullsized":"2024-04-25T19:04:25.070256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:58.847776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040396+0000","last_change":"2024-04-25T19:04:24.027966+0000","last_active":"2024-04-25T19:04:25.040396+0000","last_peered":"2024-04-25T19:04:25.040396+0000","last_clean":"2024-04-25T19:04:25.040396+0000","last_became_active":"2024-04-25T19:04:24.026879+0000","last_became_peered":"2024-04-25T19:04:24.026879+0000","last_unstale":"2024-04-25T19:04:25.040396+0000","last_undegraded":"2024-04-25T19:04:25.040396+0000","last_fullsized":"2024-04-25T19:04:25.040396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:55.866046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183809+0000","last_change":"2024-04-25T19:04:24.036651+0000","last_active":"2024-04-25T19:04:25.183809+0000","last_peered":"2024-04-25T19:04:25.183809+0000","last_clean":"2024-04-25T19:04:25.183809+0000","last_became_active":"2024-04-25T19:04:24.035699+0000","last_became_peered":"2024-04-25T19:04:24.035699+0000","last_unstale":"2024-04-25T19:04:25.183809+0000","last_undegraded":"2024-04-25T19:04:25.183809+0000","last_fullsized":"2024-04-25T19:04:25.183809+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:24.056995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040400+0000","last_change":"2024-04-25T19:04:24.044478+0000","last_active":"2024-04-25T19:04:25.040400+0000","last_peered":"2024-04-25T19:04:25.040400+0000","last_clean":"2024-04-25T19:04:25.040400+0000","last_became_active":"2024-04-25T19:04:24.044148+0000","last_became_peered":"2024-04-25T19:04:24.044148+0000","last_unstale":"2024-04-25T19:04:25.040400+0000","last_undegraded":"2024-04-25T19:04:25.040400+0000","last_fullsized":"2024-04-25T19:04:25.040400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:11.952061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.071005+0000","last_change":"2024-04-25T19:04:24.051649+0000","last_active":"2024-04-25T19:04:25.071005+0000","last_peered":"2024-04-25T19:04:25.071005+0000","last_clean":"2024-04-25T19:04:25.071005+0000","last_became_active":"2024-04-25T19:04:24.051236+0000","last_became_peered":"2024-04-25T19:04:24.051236+0000","last_unstale":"2024-04-25T19:04:25.071005+0000","last_undegraded":"2024-04-25T19:04:25.071005+0000","last_fullsized":"2024-04-25T19:04:25.071005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:24.056995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184391+0000","last_change":"2024-04-25T19:04:24.051942+0000","last_active":"2024-04-25T19:04:25.184391+0000","last_peered":"2024-04-25T19:04:25.184391+0000","last_clean":"2024-04-25T19:04:25.184391+0000","last_became_active":"2024-04-25T19:04:24.051617+0000","last_became_peered":"2024-04-25T19:04:24.051617+0000","last_unstale":"2024-04-25T19:04:25.184391+0000","last_undegraded":"2024-04-25T19:04:25.184391+0000","last_fullsized":"2024-04-25T19:04:25.184391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:45.354380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.075009+0000","last_change":"2024-04-25T19:04:24.041593+0000","last_active":"2024-04-25T19:04:25.075009+0000","last_peered":"2024-04-25T19:04:25.075009+0000","last_clean":"2024-04-25T19:04:25.075009+0000","last_became_active":"2024-04-25T19:04:24.041263+0000","last_became_peered":"2024-04-25T19:04:24.041263+0000","last_unstale":"2024-04-25T19:04:25.075009+0000","last_undegraded":"2024-04-25T19:04:25.075009+0000","last_fullsized":"2024-04-25T19:04:25.075009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:57.599213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070478+0000","last_change":"2024-04-25T19:04:24.031850+0000","last_active":"2024-04-25T19:04:25.070478+0000","last_peered":"2024-04-25T19:04:25.070478+0000","last_clean":"2024-04-25T19:04:25.070478+0000","last_became_active":"2024-04-25T19:04:24.031518+0000","last_became_peered":"2024-04-25T19:04:24.031518+0000","last_unstale":"2024-04-25T19:04:25.070478+0000","last_undegraded":"2024-04-25T19:04:25.070478+0000","last_fullsized":"2024-04-25T19:04:25.070478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:16.837527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184149+0000","last_change":"2024-04-25T19:04:24.043290+0000","last_active":"2024-04-25T19:04:25.184149+0000","last_peered":"2024-04-25T19:04:25.184149+0000","last_clean":"2024-04-25T19:04:25.184149+0000","last_became_active":"2024-04-25T19:04:24.042941+0000","last_became_peered":"2024-04-25T19:04:24.042941+0000","last_unstale":"2024-04-25T19:04:25.184149+0000","last_undegraded":"2024-04-25T19:04:25.184149+0000","last_fullsized":"2024-04-25T19:04:25.184149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.399479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185367+0000","last_change":"2024-04-25T19:04:24.051020+0000","last_active":"2024-04-25T19:04:25.185367+0000","last_peered":"2024-04-25T19:04:25.185367+0000","last_clean":"2024-04-25T19:04:25.185367+0000","last_became_active":"2024-04-25T19:04:24.050660+0000","last_became_peered":"2024-04-25T19:04:24.050660+0000","last_unstale":"2024-04-25T19:04:25.185367+0000","last_undegraded":"2024-04-25T19:04:25.185367+0000","last_fullsized":"2024-04-25T19:04:25.185367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:49.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037445+0000","last_change":"2024-04-25T19:04:24.043200+0000","last_active":"2024-04-25T19:04:25.037445+0000","last_peered":"2024-04-25T19:04:25.037445+0000","last_clean":"2024-04-25T19:04:25.037445+0000","last_became_active":"2024-04-25T19:04:24.042804+0000","last_became_peered":"2024-04-25T19:04:24.042804+0000","last_unstale":"2024-04-25T19:04:25.037445+0000","last_undegraded":"2024-04-25T19:04:25.037445+0000","last_fullsized":"2024-04-25T19:04:25.037445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:23.174340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183798+0000","last_change":"2024-04-25T19:04:24.036275+0000","last_active":"2024-04-25T19:04:25.183798+0000","last_peered":"2024-04-25T19:04:25.183798+0000","last_clean":"2024-04-25T19:04:25.183798+0000","last_became_active":"2024-04-25T19:04:24.035922+0000","last_became_peered":"2024-04-25T19:04:24.035922+0000","last_unstale":"2024-04-25T19:04:25.183798+0000","last_undegraded":"2024-04-25T19:04:25.183798+0000","last_fullsized":"2024-04-25T19:04:25.183798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:06.590035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038561+0000","last_change":"2024-04-25T19:04:24.057378+0000","last_active":"2024-04-25T19:04:25.038561+0000","last_peered":"2024-04-25T19:04:25.038561+0000","last_clean":"2024-04-25T19:04:25.038561+0000","last_became_active":"2024-04-25T19:04:24.056932+0000","last_became_peered":"2024-04-25T19:04:24.056932+0000","last_unstale":"2024-04-25T19:04:25.038561+0000","last_undegraded":"2024-04-25T19:04:25.038561+0000","last_fullsized":"2024-04-25T19:04:25.038561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:44.210425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039500+0000","last_change":"2024-04-25T19:04:24.051453+0000","last_active":"2024-04-25T19:04:25.039500+0000","last_peered":"2024-04-25T19:04:25.039500+0000","last_clean":"2024-04-25T19:04:25.039500+0000","last_became_active":"2024-04-25T19:04:24.050570+0000","last_became_peered":"2024-04-25T19:04:24.050570+0000","last_unstale":"2024-04-25T19:04:25.039500+0000","last_undegraded":"2024-04-25T19:04:25.039500+0000","last_fullsized":"2024-04-25T19:04:25.039500+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:08.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.187099+0000","last_change":"2024-04-25T19:04:24.032592+0000","last_active":"2024-04-25T19:04:25.187099+0000","last_peered":"2024-04-25T19:04:25.187099+0000","last_clean":"2024-04-25T19:04:25.187099+0000","last_became_active":"2024-04-25T19:04:24.032252+0000","last_became_peered":"2024-04-25T19:04:24.032252+0000","last_unstale":"2024-04-25T19:04:25.187099+0000","last_undegraded":"2024-04-25T19:04:25.187099+0000","last_fullsized":"2024-04-25T19:04:25.187099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:16:16.893085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074607+0000","last_change":"2024-04-25T19:04:24.055889+0000","last_active":"2024-04-25T19:04:25.074607+0000","last_peered":"2024-04-25T19:04:25.074607+0000","last_clean":"2024-04-25T19:04:25.074607+0000","last_became_active":"2024-04-25T19:04:24.055571+0000","last_became_peered":"2024-04-25T19:04:24.055571+0000","last_unstale":"2024-04-25T19:04:25.074607+0000","last_undegraded":"2024-04-25T19:04:25.074607+0000","last_fullsized":"2024-04-25T19:04:25.074607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:46.897360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074528+0000","last_change":"2024-04-25T19:04:24.046608+0000","last_active":"2024-04-25T19:04:25.074528+0000","last_peered":"2024-04-25T19:04:25.074528+0000","last_clean":"2024-04-25T19:04:25.074528+0000","last_became_active":"2024-04-25T19:04:24.046277+0000","last_became_peered":"2024-04-25T19:04:24.046277+0000","last_unstale":"2024-04-25T19:04:25.074528+0000","last_undegraded":"2024-04-25T19:04:25.074528+0000","last_fullsized":"2024-04-25T19:04:25.074528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:22.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074121+0000","last_change":"2024-04-25T19:04:24.041171+0000","last_active":"2024-04-25T19:04:25.074121+0000","last_peered":"2024-04-25T19:04:25.074121+0000","last_clean":"2024-04-25T19:04:25.074121+0000","last_became_active":"2024-04-25T19:04:24.038983+0000","last_became_peered":"2024-04-25T19:04:24.038983+0000","last_unstale":"2024-04-25T19:04:25.074121+0000","last_undegraded":"2024-04-25T19:04:25.074121+0000","last_fullsized":"2024-04-25T19:04:25.074121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:19.261202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037627+0000","last_change":"2024-04-25T19:04:24.032858+0000","last_active":"2024-04-25T19:04:25.037627+0000","last_peered":"2024-04-25T19:04:25.037627+0000","last_clean":"2024-04-25T19:04:25.037627+0000","last_became_active":"2024-04-25T19:04:24.032508+0000","last_became_peered":"2024-04-25T19:04:24.032508+0000","last_unstale":"2024-04-25T19:04:25.037627+0000","last_undegraded":"2024-04-25T19:04:25.037627+0000","last_fullsized":"2024-04-25T19:04:25.037627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:20.048729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043601+0000","last_change":"2024-04-25T19:04:24.035968+0000","last_active":"2024-04-25T19:04:25.043601+0000","last_peered":"2024-04-25T19:04:25.043601+0000","last_clean":"2024-04-25T19:04:25.043601+0000","last_became_active":"2024-04-25T19:04:24.035536+0000","last_became_peered":"2024-04-25T19:04:24.035536+0000","last_unstale":"2024-04-25T19:04:25.043601+0000","last_undegraded":"2024-04-25T19:04:25.043601+0000","last_fullsized":"2024-04-25T19:04:25.043601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:02.078455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185862+0000","last_change":"2024-04-25T19:04:24.037147+0000","last_active":"2024-04-25T19:04:25.185862+0000","last_peered":"2024-04-25T19:04:25.185862+0000","last_clean":"2024-04-25T19:04:25.185862+0000","last_became_active":"2024-04-25T19:04:24.034691+0000","last_became_peered":"2024-04-25T19:04:24.034691+0000","last_unstale":"2024-04-25T19:04:25.185862+0000","last_undegraded":"2024-04-25T19:04:25.185862+0000","last_fullsized":"2024-04-25T19:04:25.185862+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:58.504940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041668+0000","last_change":"2024-04-25T19:04:24.038466+0000","last_active":"2024-04-25T19:04:25.041668+0000","last_peered":"2024-04-25T19:04:25.041668+0000","last_clean":"2024-04-25T19:04:25.041668+0000","last_became_active":"2024-04-25T19:04:24.034200+0000","last_became_peered":"2024-04-25T19:04:24.034200+0000","last_unstale":"2024-04-25T19:04:25.041668+0000","last_undegraded":"2024-04-25T19:04:25.041668+0000","last_fullsized":"2024-04-25T19:04:25.041668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:37.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.075122+0000","last_change":"2024-04-25T19:04:24.038502+0000","last_active":"2024-04-25T19:04:25.075122+0000","last_peered":"2024-04-25T19:04:25.075122+0000","last_clean":"2024-04-25T19:04:25.075122+0000","last_became_active":"2024-04-25T19:04:24.037920+0000","last_became_peered":"2024-04-25T19:04:24.037920+0000","last_unstale":"2024-04-25T19:04:25.075122+0000","last_undegraded":"2024-04-25T19:04:25.075122+0000","last_fullsized":"2024-04-25T19:04:25.075122+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:25.998215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184273+0000","last_change":"2024-04-25T19:04:24.036430+0000","last_active":"2024-04-25T19:04:25.184273+0000","last_peered":"2024-04-25T19:04:25.184273+0000","last_clean":"2024-04-25T19:04:25.184273+0000","last_became_active":"2024-04-25T19:04:24.036098+0000","last_became_peered":"2024-04-25T19:04:24.036098+0000","last_unstale":"2024-04-25T19:04:25.184273+0000","last_undegraded":"2024-04-25T19:04:25.184273+0000","last_fullsized":"2024-04-25T19:04:25.184273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:41.182760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038564+0000","last_change":"2024-04-25T19:04:24.048424+0000","last_active":"2024-04-25T19:04:25.038564+0000","last_peered":"2024-04-25T19:04:25.038564+0000","last_clean":"2024-04-25T19:04:25.038564+0000","last_became_active":"2024-04-25T19:04:24.048045+0000","last_became_peered":"2024-04-25T19:04:24.048045+0000","last_unstale":"2024-04-25T19:04:25.038564+0000","last_undegraded":"2024-04-25T19:04:25.038564+0000","last_fullsized":"2024-04-25T19:04:25.038564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:29.267784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038105+0000","last_change":"2024-04-25T19:04:24.043795+0000","last_active":"2024-04-25T19:04:25.038105+0000","last_peered":"2024-04-25T19:04:25.038105+0000","last_clean":"2024-04-25T19:04:25.038105+0000","last_became_active":"2024-04-25T19:04:24.042635+0000","last_became_peered":"2024-04-25T19:04:24.042635+0000","last_unstale":"2024-04-25T19:04:25.038105+0000","last_undegraded":"2024-04-25T19:04:25.038105+0000","last_fullsized":"2024-04-25T19:04:25.038105+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:37.549234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183819+0000","last_change":"2024-04-25T19:04:24.052575+0000","last_active":"2024-04-25T19:04:25.183819+0000","last_peered":"2024-04-25T19:04:25.183819+0000","last_clean":"2024-04-25T19:04:25.183819+0000","last_became_active":"2024-04-25T19:04:24.049724+0000","last_became_peered":"2024-04-25T19:04:24.049724+0000","last_unstale":"2024-04-25T19:04:25.183819+0000","last_undegraded":"2024-04-25T19:04:25.183819+0000","last_fullsized":"2024-04-25T19:04:25.183819+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:48.426760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041507+0000","last_change":"2024-04-25T19:04:24.045133+0000","last_active":"2024-04-25T19:04:25.041507+0000","last_peered":"2024-04-25T19:04:25.041507+0000","last_clean":"2024-04-25T19:04:25.041507+0000","last_became_active":"2024-04-25T19:04:24.044721+0000","last_became_peered":"2024-04-25T19:04:24.044721+0000","last_unstale":"2024-04-25T19:04:25.041507+0000","last_undegraded":"2024-04-25T19:04:25.041507+0000","last_fullsized":"2024-04-25T19:04:25.041507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:27.067361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039174+0000","last_change":"2024-04-25T19:04:24.040985+0000","last_active":"2024-04-25T19:04:25.039174+0000","last_peered":"2024-04-25T19:04:25.039174+0000","last_clean":"2024-04-25T19:04:25.039174+0000","last_became_active":"2024-04-25T19:04:24.040574+0000","last_became_peered":"2024-04-25T19:04:24.040574+0000","last_unstale":"2024-04-25T19:04:25.039174+0000","last_undegraded":"2024-04-25T19:04:25.039174+0000","last_fullsized":"2024-04-25T19:04:25.039174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:03.124793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185788+0000","last_change":"2024-04-25T19:04:23.030917+0000","last_active":"2024-04-25T19:04:25.185788+0000","last_peered":"2024-04-25T19:04:25.185788+0000","last_clean":"2024-04-25T19:04:25.185788+0000","last_became_active":"2024-04-25T19:04:23.030492+0000","last_became_peered":"2024-04-25T19:04:23.030492+0000","last_unstale":"2024-04-25T19:04:25.185788+0000","last_undegraded":"2024-04-25T19:04:25.185788+0000","last_fullsized":"2024-04-25T19:04:25.185788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:36:44.339184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041281+0000","last_change":"2024-04-25T19:04:24.040734+0000","last_active":"2024-04-25T19:04:25.041281+0000","last_peered":"2024-04-25T19:04:25.041281+0000","last_clean":"2024-04-25T19:04:25.041281+0000","last_became_active":"2024-04-25T19:04:24.040397+0000","last_became_peered":"2024-04-25T19:04:24.040397+0000","last_unstale":"2024-04-25T19:04:25.041281+0000","last_undegraded":"2024-04-25T19:04:25.041281+0000","last_fullsized":"2024-04-25T19:04:25.041281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:17.487495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043204+0000","last_change":"2024-04-25T19:04:23.035885+0000","last_active":"2024-04-25T19:04:25.043204+0000","last_peered":"2024-04-25T19:04:25.043204+0000","last_clean":"2024-04-25T19:04:25.043204+0000","last_became_active":"2024-04-25T19:04:23.035094+0000","last_became_peered":"2024-04-25T19:04:23.035094+0000","last_unstale":"2024-04-25T19:04:25.043204+0000","last_undegraded":"2024-04-25T19:04:25.043204+0000","last_fullsized":"2024-04-25T19:04:25.043204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:29.579519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039728+0000","last_change":"2024-04-25T19:04:24.042473+0000","last_active":"2024-04-25T19:04:25.039728+0000","last_peered":"2024-04-25T19:04:25.039728+0000","last_clean":"2024-04-25T19:04:25.039728+0000","last_became_active":"2024-04-25T19:04:24.038290+0000","last_became_peered":"2024-04-25T19:04:24.038290+0000","last_unstale":"2024-04-25T19:04:25.039728+0000","last_undegraded":"2024-04-25T19:04:25.039728+0000","last_fullsized":"2024-04-25T19:04:25.039728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:41.575772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043238+0000","last_change":"2024-04-25T19:04:23.038315+0000","last_active":"2024-04-25T19:04:25.043238+0000","last_peered":"2024-04-25T19:04:25.043238+0000","last_clean":"2024-04-25T19:04:25.043238+0000","last_became_active":"2024-04-25T19:04:23.036771+0000","last_became_peered":"2024-04-25T19:04:23.036771+0000","last_unstale":"2024-04-25T19:04:25.043238+0000","last_undegraded":"2024-04-25T19:04:25.043238+0000","last_fullsized":"2024-04-25T19:04:25.043238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:13.998986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042085+0000","last_change":"2024-04-25T19:04:24.043120+0000","last_active":"2024-04-25T19:04:25.042085+0000","last_peered":"2024-04-25T19:04:25.042085+0000","last_clean":"2024-04-25T19:04:25.042085+0000","last_became_active":"2024-04-25T19:04:24.042756+0000","last_became_peered":"2024-04-25T19:04:24.042756+0000","last_unstale":"2024-04-25T19:04:25.042085+0000","last_undegraded":"2024-04-25T19:04:25.042085+0000","last_fullsized":"2024-04-25T19:04:25.042085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:17:35.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185640+0000","last_change":"2024-04-25T19:04:23.032468+0000","last_active":"2024-04-25T19:04:25.185640+0000","last_peered":"2024-04-25T19:04:25.185640+0000","last_clean":"2024-04-25T19:04:25.185640+0000","last_became_active":"2024-04-25T19:04:23.031706+0000","last_became_peered":"2024-04-25T19:04:23.031706+0000","last_unstale":"2024-04-25T19:04:25.185640+0000","last_undegraded":"2024-04-25T19:04:25.185640+0000","last_fullsized":"2024-04-25T19:04:25.185640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:05.384283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042566+0000","last_change":"2024-04-25T19:04:24.040806+0000","last_active":"2024-04-25T19:04:25.042566+0000","last_peered":"2024-04-25T19:04:25.042566+0000","last_clean":"2024-04-25T19:04:25.042566+0000","last_became_active":"2024-04-25T19:04:24.038096+0000","last_became_peered":"2024-04-25T19:04:24.038096+0000","last_unstale":"2024-04-25T19:04:25.042566+0000","last_undegraded":"2024-04-25T19:04:25.042566+0000","last_fullsized":"2024-04-25T19:04:25.042566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:59.757543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185475+0000","last_change":"2024-04-25T19:04:23.035962+0000","last_active":"2024-04-25T19:04:25.185475+0000","last_peered":"2024-04-25T19:04:25.185475+0000","last_clean":"2024-04-25T19:04:25.185475+0000","last_became_active":"2024-04-25T19:04:23.035629+0000","last_became_peered":"2024-04-25T19:04:23.035629+0000","last_unstale":"2024-04-25T19:04:25.185475+0000","last_undegraded":"2024-04-25T19:04:25.185475+0000","last_fullsized":"2024-04-25T19:04:25.185475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:47.411565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074114+0000","last_change":"2024-04-25T19:04:24.029100+0000","last_active":"2024-04-25T19:04:25.074114+0000","last_peered":"2024-04-25T19:04:25.074114+0000","last_clean":"2024-04-25T19:04:25.074114+0000","last_became_active":"2024-04-25T19:04:24.028647+0000","last_became_peered":"2024-04-25T19:04:24.028647+0000","last_unstale":"2024-04-25T19:04:25.074114+0000","last_undegraded":"2024-04-25T19:04:25.074114+0000","last_fullsized":"2024-04-25T19:04:25.074114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:41.902197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183910+0000","last_change":"2024-04-25T19:04:23.042183+0000","last_active":"2024-04-25T19:04:25.183910+0000","last_peered":"2024-04-25T19:04:25.183910+0000","last_clean":"2024-04-25T19:04:25.183910+0000","last_became_active":"2024-04-25T19:04:23.041863+0000","last_became_peered":"2024-04-25T19:04:23.041863+0000","last_unstale":"2024-04-25T19:04:25.183910+0000","last_undegraded":"2024-04-25T19:04:25.183910+0000","last_fullsized":"2024-04-25T19:04:25.183910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:46.772247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038584+0000","last_change":"2024-04-25T19:04:24.051602+0000","last_active":"2024-04-25T19:04:25.038584+0000","last_peered":"2024-04-25T19:04:25.038584+0000","last_clean":"2024-04-25T19:04:25.038584+0000","last_became_active":"2024-04-25T19:04:24.051089+0000","last_became_peered":"2024-04-25T19:04:24.051089+0000","last_unstale":"2024-04-25T19:04:25.038584+0000","last_undegraded":"2024-04-25T19:04:25.038584+0000","last_fullsized":"2024-04-25T19:04:25.038584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:12.011664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039553+0000","last_change":"2024-04-25T19:04:23.035655+0000","last_active":"2024-04-25T19:04:25.039553+0000","last_peered":"2024-04-25T19:04:25.039553+0000","last_clean":"2024-04-25T19:04:25.039553+0000","last_became_active":"2024-04-25T19:04:23.034401+0000","last_became_peered":"2024-04-25T19:04:23.034401+0000","last_unstale":"2024-04-25T19:04:25.039553+0000","last_undegraded":"2024-04-25T19:04:25.039553+0000","last_fullsized":"2024-04-25T19:04:25.039553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:26.969350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040344+0000","last_change":"2024-04-25T19:04:24.035929+0000","last_active":"2024-04-25T19:04:25.040344+0000","last_peered":"2024-04-25T19:04:25.040344+0000","last_clean":"2024-04-25T19:04:25.040344+0000","last_became_active":"2024-04-25T19:04:24.035598+0000","last_became_peered":"2024-04-25T19:04:24.035598+0000","last_unstale":"2024-04-25T19:04:25.040344+0000","last_undegraded":"2024-04-25T19:04:25.040344+0000","last_fullsized":"2024-04-25T19:04:25.040344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:16.266368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183744+0000","last_change":"2024-04-25T19:04:23.038320+0000","last_active":"2024-04-25T19:04:25.183744+0000","last_peered":"2024-04-25T19:04:25.183744+0000","last_clean":"2024-04-25T19:04:25.183744+0000","last_became_active":"2024-04-25T19:04:23.037799+0000","last_became_peered":"2024-04-25T19:04:23.037799+0000","last_unstale":"2024-04-25T19:04:25.183744+0000","last_undegraded":"2024-04-25T19:04:25.183744+0000","last_fullsized":"2024-04-25T19:04:25.183744+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:02:25.923453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043459+0000","last_change":"2024-04-25T19:04:24.042240+0000","last_active":"2024-04-25T19:04:25.043459+0000","last_peered":"2024-04-25T19:04:25.043459+0000","last_clean":"2024-04-25T19:04:25.043459+0000","last_became_active":"2024-04-25T19:04:24.041829+0000","last_became_peered":"2024-04-25T19:04:24.041829+0000","last_unstale":"2024-04-25T19:04:25.043459+0000","last_undegraded":"2024-04-25T19:04:25.043459+0000","last_fullsized":"2024-04-25T19:04:25.043459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:09:51.783876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041556+0000","last_change":"2024-04-25T19:04:23.033714+0000","last_active":"2024-04-25T19:04:25.041556+0000","last_peered":"2024-04-25T19:04:25.041556+0000","last_clean":"2024-04-25T19:04:25.041556+0000","last_became_active":"2024-04-25T19:04:23.032605+0000","last_became_peered":"2024-04-25T19:04:23.032605+0000","last_unstale":"2024-04-25T19:04:25.041556+0000","last_undegraded":"2024-04-25T19:04:25.041556+0000","last_fullsized":"2024-04-25T19:04:25.041556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:48.922241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185746+0000","last_change":"2024-04-25T19:04:24.043018+0000","last_active":"2024-04-25T19:04:25.185746+0000","last_peered":"2024-04-25T19:04:25.185746+0000","last_clean":"2024-04-25T19:04:25.185746+0000","last_became_active":"2024-04-25T19:04:24.039687+0000","last_became_peered":"2024-04-25T19:04:24.039687+0000","last_unstale":"2024-04-25T19:04:25.185746+0000","last_undegraded":"2024-04-25T19:04:25.185746+0000","last_fullsized":"2024-04-25T19:04:25.185746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:24.303094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039421+0000","last_change":"2024-04-25T19:04:23.036996+0000","last_active":"2024-04-25T19:04:25.039421+0000","last_peered":"2024-04-25T19:04:25.039421+0000","last_clean":"2024-04-25T19:04:25.039421+0000","last_became_active":"2024-04-25T19:04:23.036583+0000","last_became_peered":"2024-04-25T19:04:23.036583+0000","last_unstale":"2024-04-25T19:04:25.039421+0000","last_undegraded":"2024-04-25T19:04:25.039421+0000","last_fullsized":"2024-04-25T19:04:25.039421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:02.734018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043210+0000","last_change":"2024-04-25T19:04:24.042595+0000","last_active":"2024-04-25T19:04:25.043210+0000","last_peered":"2024-04-25T19:04:25.043210+0000","last_clean":"2024-04-25T19:04:25.043210+0000","last_became_active":"2024-04-25T19:04:24.037589+0000","last_became_peered":"2024-04-25T19:04:24.037589+0000","last_unstale":"2024-04-25T19:04:25.043210+0000","last_undegraded":"2024-04-25T19:04:25.043210+0000","last_fullsized":"2024-04-25T19:04:25.043210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.776836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037955+0000","last_change":"2024-04-25T19:04:23.042106+0000","last_active":"2024-04-25T19:04:25.037955+0000","last_peered":"2024-04-25T19:04:25.037955+0000","last_clean":"2024-04-25T19:04:25.037955+0000","last_became_active":"2024-04-25T19:04:23.041280+0000","last_became_peered":"2024-04-25T19:04:23.041280+0000","last_unstale":"2024-04-25T19:04:25.037955+0000","last_undegraded":"2024-04-25T19:04:25.037955+0000","last_fullsized":"2024-04-25T19:04:25.037955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:45.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184517+0000","last_change":"2024-04-25T19:04:24.034681+0000","last_active":"2024-04-25T19:04:25.184517+0000","last_peered":"2024-04-25T19:04:25.184517+0000","last_clean":"2024-04-25T19:04:25.184517+0000","last_became_active":"2024-04-25T19:04:24.034337+0000","last_became_peered":"2024-04-25T19:04:24.034337+0000","last_unstale":"2024-04-25T19:04:25.184517+0000","last_undegraded":"2024-04-25T19:04:25.184517+0000","last_fullsized":"2024-04-25T19:04:25.184517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:15.014182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042898+0000","last_change":"2024-04-25T19:04:23.028456+0000","last_active":"2024-04-25T19:04:25.042898+0000","last_peered":"2024-04-25T19:04:25.042898+0000","last_clean":"2024-04-25T19:04:25.042898+0000","last_became_active":"2024-04-25T19:04:23.028022+0000","last_became_peered":"2024-04-25T19:04:23.028022+0000","last_unstale":"2024-04-25T19:04:25.042898+0000","last_undegraded":"2024-04-25T19:04:25.042898+0000","last_fullsized":"2024-04-25T19:04:25.042898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:19.295776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043543+0000","last_change":"2024-04-25T19:04:24.035501+0000","last_active":"2024-04-25T19:04:25.043543+0000","last_peered":"2024-04-25T19:04:25.043543+0000","last_clean":"2024-04-25T19:04:25.043543+0000","last_became_active":"2024-04-25T19:04:24.035180+0000","last_became_peered":"2024-04-25T19:04:24.035180+0000","last_unstale":"2024-04-25T19:04:25.043543+0000","last_undegraded":"2024-04-25T19:04:25.043543+0000","last_fullsized":"2024-04-25T19:04:25.043543+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:47.657249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041102+0000","last_change":"2024-04-25T19:04:23.039931+0000","last_active":"2024-04-25T19:04:25.041102+0000","last_peered":"2024-04-25T19:04:25.041102+0000","last_clean":"2024-04-25T19:04:25.041102+0000","last_became_active":"2024-04-25T19:04:23.038962+0000","last_became_peered":"2024-04-25T19:04:23.038962+0000","last_unstale":"2024-04-25T19:04:25.041102+0000","last_undegraded":"2024-04-25T19:04:25.041102+0000","last_fullsized":"2024-04-25T19:04:25.041102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:48.274232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041078+0000","last_change":"2024-04-25T19:04:24.042094+0000","last_active":"2024-04-25T19:04:25.041078+0000","last_peered":"2024-04-25T19:04:25.041078+0000","last_clean":"2024-04-25T19:04:25.041078+0000","last_became_active":"2024-04-25T19:04:24.041343+0000","last_became_peered":"2024-04-25T19:04:24.041343+0000","last_unstale":"2024-04-25T19:04:25.041078+0000","last_undegraded":"2024-04-25T19:04:25.041078+0000","last_fullsized":"2024-04-25T19:04:25.041078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:41.267557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040771+0000","last_change":"2024-04-25T19:04:23.039863+0000","last_active":"2024-04-25T19:04:25.040771+0000","last_peered":"2024-04-25T19:04:25.040771+0000","last_clean":"2024-04-25T19:04:25.040771+0000","last_became_active":"2024-04-25T19:04:23.039505+0000","last_became_peered":"2024-04-25T19:04:23.039505+0000","last_unstale":"2024-04-25T19:04:25.040771+0000","last_undegraded":"2024-04-25T19:04:25.040771+0000","last_fullsized":"2024-04-25T19:04:25.040771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.507979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074530+0000","last_change":"2024-04-25T19:04:24.036714+0000","last_active":"2024-04-25T19:04:25.074530+0000","last_peered":"2024-04-25T19:04:25.074530+0000","last_clean":"2024-04-25T19:04:25.074530+0000","last_became_active":"2024-04-25T19:04:24.034451+0000","last_became_peered":"2024-04-25T19:04:24.034451+0000","last_unstale":"2024-04-25T19:04:25.074530+0000","last_undegraded":"2024-04-25T19:04:25.074530+0000","last_fullsized":"2024-04-25T19:04:25.074530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:58:21.739287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037687+0000","last_change":"2024-04-25T19:04:23.029445+0000","last_active":"2024-04-25T19:04:25.037687+0000","last_peered":"2024-04-25T19:04:25.037687+0000","last_clean":"2024-04-25T19:04:25.037687+0000","last_became_active":"2024-04-25T19:04:23.029030+0000","last_became_peered":"2024-04-25T19:04:23.029030+0000","last_unstale":"2024-04-25T19:04:25.037687+0000","last_undegraded":"2024-04-25T19:04:25.037687+0000","last_fullsized":"2024-04-25T19:04:25.037687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:17.087167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070321+0000","last_change":"2024-04-25T19:04:24.026128+0000","last_active":"2024-04-25T19:04:25.070321+0000","last_peered":"2024-04-25T19:04:25.070321+0000","last_clean":"2024-04-25T19:04:25.070321+0000","last_became_active":"2024-04-25T19:04:24.025705+0000","last_became_peered":"2024-04-25T19:04:24.025705+0000","last_unstale":"2024-04-25T19:04:25.070321+0000","last_undegraded":"2024-04-25T19:04:25.070321+0000","last_fullsized":"2024-04-25T19:04:25.070321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:02.088683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.185263+0000","last_change":"2024-04-25T19:04:23.042382+0000","last_active":"2024-04-25T19:04:25.185263+0000","last_peered":"2024-04-25T19:04:25.185263+0000","last_clean":"2024-04-25T19:04:25.185263+0000","last_became_active":"2024-04-25T19:04:23.042083+0000","last_became_peered":"2024-04-25T19:04:23.042083+0000","last_unstale":"2024-04-25T19:04:25.185263+0000","last_undegraded":"2024-04-25T19:04:25.185263+0000","last_fullsized":"2024-04-25T19:04:25.185263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:40.167565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040951+0000","last_change":"2024-04-25T19:04:24.037283+0000","last_active":"2024-04-25T19:04:25.040951+0000","last_peered":"2024-04-25T19:04:25.040951+0000","last_clean":"2024-04-25T19:04:25.040951+0000","last_became_active":"2024-04-25T19:04:24.036962+0000","last_became_peered":"2024-04-25T19:04:24.036962+0000","last_unstale":"2024-04-25T19:04:25.040951+0000","last_undegraded":"2024-04-25T19:04:25.040951+0000","last_fullsized":"2024-04-25T19:04:25.040951+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:50.762653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184475+0000","last_change":"2024-04-25T19:04:23.035593+0000","last_active":"2024-04-25T19:04:25.184475+0000","last_peered":"2024-04-25T19:04:25.184475+0000","last_clean":"2024-04-25T19:04:25.184475+0000","last_became_active":"2024-04-25T19:04:23.035271+0000","last_became_peered":"2024-04-25T19:04:23.035271+0000","last_unstale":"2024-04-25T19:04:25.184475+0000","last_undegraded":"2024-04-25T19:04:25.184475+0000","last_fullsized":"2024-04-25T19:04:25.184475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184505+0000","last_change":"2024-04-25T19:04:24.038008+0000","last_active":"2024-04-25T19:04:25.184505+0000","last_peered":"2024-04-25T19:04:25.184505+0000","last_clean":"2024-04-25T19:04:25.184505+0000","last_became_active":"2024-04-25T19:04:24.037600+0000","last_became_peered":"2024-04-25T19:04:24.037600+0000","last_unstale":"2024-04-25T19:04:25.184505+0000","last_undegraded":"2024-04-25T19:04:25.184505+0000","last_fullsized":"2024-04-25T19:04:25.184505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:13.502392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039621+0000","last_change":"2024-04-25T19:04:23.040668+0000","last_active":"2024-04-25T19:04:25.039621+0000","last_peered":"2024-04-25T19:04:25.039621+0000","last_clean":"2024-04-25T19:04:25.039621+0000","last_became_active":"2024-04-25T19:04:23.040355+0000","last_became_peered":"2024-04-25T19:04:23.040355+0000","last_unstale":"2024-04-25T19:04:25.039621+0000","last_undegraded":"2024-04-25T19:04:25.039621+0000","last_fullsized":"2024-04-25T19:04:25.039621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:58:20.724091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.184442+0000","last_change":"2024-04-25T19:04:24.044989+0000","last_active":"2024-04-25T19:04:25.184442+0000","last_peered":"2024-04-25T19:04:25.184442+0000","last_clean":"2024-04-25T19:04:25.184442+0000","last_became_active":"2024-04-25T19:04:24.044592+0000","last_became_peered":"2024-04-25T19:04:24.044592+0000","last_unstale":"2024-04-25T19:04:25.184442+0000","last_undegraded":"2024-04-25T19:04:25.184442+0000","last_fullsized":"2024-04-25T19:04:25.184442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:41.035291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070176+0000","last_change":"2024-04-25T19:04:23.042840+0000","last_active":"2024-04-25T19:04:25.070176+0000","last_peered":"2024-04-25T19:04:25.070176+0000","last_clean":"2024-04-25T19:04:25.070176+0000","last_became_active":"2024-04-25T19:04:23.041894+0000","last_became_peered":"2024-04-25T19:04:23.041894+0000","last_unstale":"2024-04-25T19:04:25.070176+0000","last_undegraded":"2024-04-25T19:04:25.070176+0000","last_fullsized":"2024-04-25T19:04:25.070176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.187274+0000","last_change":"2024-04-25T19:04:24.037245+0000","last_active":"2024-04-25T19:04:25.187274+0000","last_peered":"2024-04-25T19:04:25.187274+0000","last_clean":"2024-04-25T19:04:25.187274+0000","last_became_active":"2024-04-25T19:04:24.036925+0000","last_became_peered":"2024-04-25T19:04:24.036925+0000","last_unstale":"2024-04-25T19:04:25.187274+0000","last_undegraded":"2024-04-25T19:04:25.187274+0000","last_fullsized":"2024-04-25T19:04:25.187274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:29.649000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.186494+0000","last_change":"2024-04-25T19:04:23.043801+0000","last_active":"2024-04-25T19:04:25.186494+0000","last_peered":"2024-04-25T19:04:25.186494+0000","last_clean":"2024-04-25T19:04:25.186494+0000","last_became_active":"2024-04-25T19:04:23.043507+0000","last_became_peered":"2024-04-25T19:04:23.043507+0000","last_unstale":"2024-04-25T19:04:25.186494+0000","last_undegraded":"2024-04-25T19:04:25.186494+0000","last_fullsized":"2024-04-25T19:04:25.186494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:23.287898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037618+0000","last_change":"2024-04-25T19:04:24.033415+0000","last_active":"2024-04-25T19:04:25.037618+0000","last_peered":"2024-04-25T19:04:25.037618+0000","last_clean":"2024-04-25T19:04:25.037618+0000","last_became_active":"2024-04-25T19:04:24.031066+0000","last_became_peered":"2024-04-25T19:04:24.031066+0000","last_unstale":"2024-04-25T19:04:25.037618+0000","last_undegraded":"2024-04-25T19:04:25.037618+0000","last_fullsized":"2024-04-25T19:04:25.037618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:59.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.187053+0000","last_change":"2024-04-25T19:04:23.037893+0000","last_active":"2024-04-25T19:04:25.187053+0000","last_peered":"2024-04-25T19:04:25.187053+0000","last_clean":"2024-04-25T19:04:25.187053+0000","last_became_active":"2024-04-25T19:04:23.036256+0000","last_became_peered":"2024-04-25T19:04:23.036256+0000","last_unstale":"2024-04-25T19:04:25.187053+0000","last_undegraded":"2024-04-25T19:04:25.187053+0000","last_fullsized":"2024-04-25T19:04:25.187053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:57.489744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039781+0000","last_change":"2024-04-25T19:04:24.045004+0000","last_active":"2024-04-25T19:04:25.039781+0000","last_peered":"2024-04-25T19:04:25.039781+0000","last_clean":"2024-04-25T19:04:25.039781+0000","last_became_active":"2024-04-25T19:04:24.044112+0000","last_became_peered":"2024-04-25T19:04:24.044112+0000","last_unstale":"2024-04-25T19:04:25.039781+0000","last_undegraded":"2024-04-25T19:04:25.039781+0000","last_fullsized":"2024-04-25T19:04:25.039781+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:25.998215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.042255+0000","last_change":"2024-04-25T19:04:23.032920+0000","last_active":"2024-04-25T19:04:25.042255+0000","last_peered":"2024-04-25T19:04:25.042255+0000","last_clean":"2024-04-25T19:04:25.042255+0000","last_became_active":"2024-04-25T19:04:23.032583+0000","last_became_peered":"2024-04-25T19:04:23.032583+0000","last_unstale":"2024-04-25T19:04:25.042255+0000","last_undegraded":"2024-04-25T19:04:25.042255+0000","last_fullsized":"2024-04-25T19:04:25.042255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:49.943413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074536+0000","last_change":"2024-04-25T19:04:24.042589+0000","last_active":"2024-04-25T19:04:25.074536+0000","last_peered":"2024-04-25T19:04:25.074536+0000","last_clean":"2024-04-25T19:04:25.074536+0000","last_became_active":"2024-04-25T19:04:24.040737+0000","last_became_peered":"2024-04-25T19:04:24.040737+0000","last_unstale":"2024-04-25T19:04:25.074536+0000","last_undegraded":"2024-04-25T19:04:25.074536+0000","last_fullsized":"2024-04-25T19:04:25.074536+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:03.749214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183865+0000","last_change":"2024-04-25T19:04:23.038543+0000","last_active":"2024-04-25T19:04:25.183865+0000","last_peered":"2024-04-25T19:04:25.183865+0000","last_clean":"2024-04-25T19:04:25.183865+0000","last_became_active":"2024-04-25T19:04:23.037940+0000","last_became_peered":"2024-04-25T19:04:23.037940+0000","last_unstale":"2024-04-25T19:04:25.183865+0000","last_undegraded":"2024-04-25T19:04:25.183865+0000","last_fullsized":"2024-04-25T19:04:25.183865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:34.225851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.040926+0000","last_change":"2024-04-25T19:04:24.042379+0000","last_active":"2024-04-25T19:04:25.040926+0000","last_peered":"2024-04-25T19:04:25.040926+0000","last_clean":"2024-04-25T19:04:25.040926+0000","last_became_active":"2024-04-25T19:04:24.041087+0000","last_became_peered":"2024-04-25T19:04:24.041087+0000","last_unstale":"2024-04-25T19:04:25.040926+0000","last_undegraded":"2024-04-25T19:04:25.040926+0000","last_fullsized":"2024-04-25T19:04:25.040926+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:41.201420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.039250+0000","last_change":"2024-04-25T19:04:23.039339+0000","last_active":"2024-04-25T19:04:25.039250+0000","last_peered":"2024-04-25T19:04:25.039250+0000","last_clean":"2024-04-25T19:04:25.039250+0000","last_became_active":"2024-04-25T19:04:23.038657+0000","last_became_peered":"2024-04-25T19:04:23.038657+0000","last_unstale":"2024-04-25T19:04:25.039250+0000","last_undegraded":"2024-04-25T19:04:25.039250+0000","last_fullsized":"2024-04-25T19:04:25.039250+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:11.204701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070530+0000","last_change":"2024-04-25T19:04:24.042564+0000","last_active":"2024-04-25T19:04:25.070530+0000","last_peered":"2024-04-25T19:04:25.070530+0000","last_clean":"2024-04-25T19:04:25.070530+0000","last_became_active":"2024-04-25T19:04:24.042170+0000","last_became_peered":"2024-04-25T19:04:24.042170+0000","last_unstale":"2024-04-25T19:04:25.070530+0000","last_undegraded":"2024-04-25T19:04:25.070530+0000","last_fullsized":"2024-04-25T19:04:25.070530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:06.590035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038630+0000","last_change":"2024-04-25T19:04:23.039240+0000","last_active":"2024-04-25T19:04:25.038630+0000","last_peered":"2024-04-25T19:04:25.038630+0000","last_clean":"2024-04-25T19:04:25.038630+0000","last_became_active":"2024-04-25T19:04:23.038906+0000","last_became_peered":"2024-04-25T19:04:23.038906+0000","last_unstale":"2024-04-25T19:04:25.038630+0000","last_undegraded":"2024-04-25T19:04:25.038630+0000","last_fullsized":"2024-04-25T19:04:25.038630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:24.983019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.075203+0000","last_change":"2024-04-25T19:04:24.042059+0000","last_active":"2024-04-25T19:04:25.075203+0000","last_peered":"2024-04-25T19:04:25.075203+0000","last_clean":"2024-04-25T19:04:25.075203+0000","last_became_active":"2024-04-25T19:04:24.041643+0000","last_became_peered":"2024-04-25T19:04:24.041643+0000","last_unstale":"2024-04-25T19:04:25.075203+0000","last_undegraded":"2024-04-25T19:04:25.075203+0000","last_fullsized":"2024-04-25T19:04:25.075203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:11.952061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070632+0000","last_change":"2024-04-25T19:04:23.037783+0000","last_active":"2024-04-25T19:04:25.070632+0000","last_peered":"2024-04-25T19:04:25.070632+0000","last_clean":"2024-04-25T19:04:25.070632+0000","last_became_active":"2024-04-25T19:04:23.037011+0000","last_became_peered":"2024-04-25T19:04:23.037011+0000","last_unstale":"2024-04-25T19:04:25.070632+0000","last_undegraded":"2024-04-25T19:04:25.070632+0000","last_fullsized":"2024-04-25T19:04:25.070632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:04.242119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.187180+0000","last_change":"2024-04-25T19:04:24.048949+0000","last_active":"2024-04-25T19:04:25.187180+0000","last_peered":"2024-04-25T19:04:25.187180+0000","last_clean":"2024-04-25T19:04:25.187180+0000","last_became_active":"2024-04-25T19:04:24.048654+0000","last_became_peered":"2024-04-25T19:04:24.048654+0000","last_unstale":"2024-04-25T19:04:25.187180+0000","last_undegraded":"2024-04-25T19:04:25.187180+0000","last_fullsized":"2024-04-25T19:04:25.187180+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:32.206903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.070323+0000","last_change":"2024-04-25T19:04:23.040808+0000","last_active":"2024-04-25T19:04:25.070323+0000","last_peered":"2024-04-25T19:04:25.070323+0000","last_clean":"2024-04-25T19:04:25.070323+0000","last_became_active":"2024-04-25T19:04:23.040480+0000","last_became_peered":"2024-04-25T19:04:23.040480+0000","last_unstale":"2024-04-25T19:04:25.070323+0000","last_undegraded":"2024-04-25T19:04:25.070323+0000","last_fullsized":"2024-04-25T19:04:25.070323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:46.772247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.069839+0000","last_change":"2024-04-25T19:04:24.027803+0000","last_active":"2024-04-25T19:04:25.069839+0000","last_peered":"2024-04-25T19:04:25.069839+0000","last_clean":"2024-04-25T19:04:25.069839+0000","last_became_active":"2024-04-25T19:04:24.027492+0000","last_became_peered":"2024-04-25T19:04:24.027492+0000","last_unstale":"2024-04-25T19:04:25.069839+0000","last_undegraded":"2024-04-25T19:04:25.069839+0000","last_fullsized":"2024-04-25T19:04:25.069839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:37.340978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041744+0000","last_change":"2024-04-25T19:04:23.039276+0000","last_active":"2024-04-25T19:04:25.041744+0000","last_peered":"2024-04-25T19:04:25.041744+0000","last_clean":"2024-04-25T19:04:25.041744+0000","last_became_active":"2024-04-25T19:04:23.038762+0000","last_became_peered":"2024-04-25T19:04:23.038762+0000","last_unstale":"2024-04-25T19:04:25.041744+0000","last_undegraded":"2024-04-25T19:04:25.041744+0000","last_fullsized":"2024-04-25T19:04:25.041744+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:23.287898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.041625+0000","last_change":"2024-04-25T19:04:24.043192+0000","last_active":"2024-04-25T19:04:25.041625+0000","last_peered":"2024-04-25T19:04:25.041625+0000","last_clean":"2024-04-25T19:04:25.041625+0000","last_became_active":"2024-04-25T19:04:24.042766+0000","last_became_peered":"2024-04-25T19:04:24.042766+0000","last_unstale":"2024-04-25T19:04:25.041625+0000","last_undegraded":"2024-04-25T19:04:25.041625+0000","last_fullsized":"2024-04-25T19:04:25.041625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:07.475303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074619+0000","last_change":"2024-04-25T19:04:23.034177+0000","last_active":"2024-04-25T19:04:25.074619+0000","last_peered":"2024-04-25T19:04:25.074619+0000","last_clean":"2024-04-25T19:04:25.074619+0000","last_became_active":"2024-04-25T19:04:23.033833+0000","last_became_peered":"2024-04-25T19:04:23.033833+0000","last_unstale":"2024-04-25T19:04:25.074619+0000","last_undegraded":"2024-04-25T19:04:25.074619+0000","last_fullsized":"2024-04-25T19:04:25.074619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:26.969350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038951+0000","last_change":"2024-04-25T19:04:24.042862+0000","last_active":"2024-04-25T19:04:25.038951+0000","last_peered":"2024-04-25T19:04:25.038951+0000","last_clean":"2024-04-25T19:04:25.038951+0000","last_became_active":"2024-04-25T19:04:24.042547+0000","last_became_peered":"2024-04-25T19:04:24.042547+0000","last_unstale":"2024-04-25T19:04:25.038951+0000","last_undegraded":"2024-04-25T19:04:25.038951+0000","last_fullsized":"2024-04-25T19:04:25.038951+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:42.437658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.037803+0000","last_change":"2024-04-25T19:04:23.039663+0000","last_active":"2024-04-25T19:04:25.037803+0000","last_peered":"2024-04-25T19:04:25.037803+0000","last_clean":"2024-04-25T19:04:25.037803+0000","last_became_active":"2024-04-25T19:04:23.038851+0000","last_became_peered":"2024-04-25T19:04:23.038851+0000","last_unstale":"2024-04-25T19:04:25.037803+0000","last_undegraded":"2024-04-25T19:04:25.037803+0000","last_fullsized":"2024-04-25T19:04:25.037803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:56.584018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074806+0000","last_change":"2024-04-25T19:04:24.036131+0000","last_active":"2024-04-25T19:04:25.074806+0000","last_peered":"2024-04-25T19:04:25.074806+0000","last_clean":"2024-04-25T19:04:25.074806+0000","last_became_active":"2024-04-25T19:04:24.033782+0000","last_became_peered":"2024-04-25T19:04:24.033782+0000","last_unstale":"2024-04-25T19:04:25.074806+0000","last_undegraded":"2024-04-25T19:04:25.074806+0000","last_fullsized":"2024-04-25T19:04:25.074806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:18.102362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.043679+0000","last_change":"2024-04-25T19:04:23.027116+0000","last_active":"2024-04-25T19:04:25.043679+0000","last_peered":"2024-04-25T19:04:25.043679+0000","last_clean":"2024-04-25T19:04:25.043679+0000","last_became_active":"2024-04-25T19:04:23.026785+0000","last_became_peered":"2024-04-25T19:04:23.026785+0000","last_unstale":"2024-04-25T19:04:25.043679+0000","last_undegraded":"2024-04-25T19:04:25.043679+0000","last_fullsized":"2024-04-25T19:04:25.043679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:57.489744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.038380+0000","last_change":"2024-04-25T19:04:24.042036+0000","last_active":"2024-04-25T19:04:25.038380+0000","last_peered":"2024-04-25T19:04:25.038380+0000","last_clean":"2024-04-25T19:04:25.038380+0000","last_became_active":"2024-04-25T19:04:24.041517+0000","last_became_peered":"2024-04-25T19:04:24.041517+0000","last_unstale":"2024-04-25T19:04:25.038380+0000","last_undegraded":"2024-04-25T19:04:25.038380+0000","last_fullsized":"2024-04-25T19:04:25.038380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:22.987829+0000","last_clean_scrub_stamp":"2024-04-25T19:04:22.987829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:53.306814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.074366+0000","last_change":"2024-04-25T19:04:23.037179+0000","last_active":"2024-04-25T19:04:25.074366+0000","last_peered":"2024-04-25T19:04:25.074366+0000","last_clean":"2024-04-25T19:04:25.074366+0000","last_became_active":"2024-04-25T19:04:23.036838+0000","last_became_peered":"2024-04-25T19:04:23.036838+0000","last_unstale":"2024-04-25T19:04:25.074366+0000","last_undegraded":"2024-04-25T19:04:25.074366+0000","last_fullsized":"2024-04-25T19:04:25.074366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:04:21.972633+0000","last_clean_scrub_stamp":"2024-04-25T19:04:21.972633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:48.274232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T19:04:25.183865+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:04:25.183865+0000","last_peered":"2024-04-25T19:04:25.183865+0000","last_clean":"2024-04-25T19:04:25.183865+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:04:25.183865+0000","last_undegraded":"2024-04-25T19:04:25.183865+0000","last_fullsized":"2024-04-25T19:04:25.183865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":785452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":196696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":784892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":194594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:04:29.241 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T19:04:29.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T19:04:29.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.558+0000 7f0da9e00640 1 Processor -- start 2024-04-25T19:04:29.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.558+0000 7f0da9e00640 1 -- start start 2024-04-25T19:04:29.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.559+0000 7f0da9e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0da40d1010 0x7f0da40cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da9e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da406dc90 0x7f0da40b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da9e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da40ceb40 con 0x7f0da40ce510 2024-04-25T19:04:29.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da9e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0da40ce920 con 0x7f0da40d1010 2024-04-25T19:04:29.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da9e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40d1420 con 0x7f0da406dc90 2024-04-25T19:04:29.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0da40ce510 0x7f0da406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36432/0 (socket says 172.21.15.123:36432) 2024-04-25T19:04:29.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da3e00640 1 -- 172.21.15.123:0/3764492701 learned_addr learned my addr 172.21.15.123:0/3764492701 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da8a00640 1 --2- 172.21.15.123:0/3764492701 >> v2:172.21.15.146:3300/0 conn(0x7f0da40d1010 0x7f0da40cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da9400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da406dc90 0x7f0da40b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 459807408 0 0) 0x7f0da40ceb40 con 0x7f0da40ce510 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.560+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d88003520 con 0x7f0da40ce510 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 971910066 0 0) 0x7f0d88003520 con 0x7f0da40ce510 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da406dc90 msgr2=0x7f0da40b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da406dc90 0x7f0da40b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 >> v2:172.21.15.146:3300/0 conn(0x7f0da40d1010 msgr2=0x7f0da40cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 --2- 172.21.15.123:0/3764492701 >> v2:172.21.15.146:3300/0 conn(0x7f0da40d1010 0x7f0da40cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da40cffb0 con 0x7f0da40ce510 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f0d8c001670 con 0x7f0da40ce510 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da8a00640 1 --2- 172.21.15.123:0/3764492701 >> v2:172.21.15.146:3300/0 conn(0x7f0da40d1010 0x7f0da40cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 >> v1:172.21.15.123:6789/0 conn(0x7f0da40ce510 legacy=0x7f0da406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d880039b0 0x7f0d88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d88004340 0x7f0d88004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88004f80 0x7f0d8800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40ce920 con 0x7f0d88004f80 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40d1420 con 0x7f0d88004340 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8800e110 con 0x7f0d880039b0 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.561+0000 7f0da9400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88004f80 0x7f0d8800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da8a00640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d880039b0 0x7f0d88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da3e00640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d88004340 0x7f0d88004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da9400640 1 -- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d880039b0 msgr2=0x7f0d88003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da9400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d880039b0 0x7f0d88003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da9400640 1 -- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d88004340 msgr2=0x7f0d88004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da9400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d88004340 0x7f0d88004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.562+0000 7f0da9400640 1 -- 172.21.15.123:0/3764492701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d88003520 con 0x7f0d88004f80 2024-04-25T19:04:29.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.563+0000 7f0da9400640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88004f80 0x7f0d8800dbd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009ac0 tx=0x7f0d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f996da6bd764395f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.563+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c036070 con 0x7f0d88004f80 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.563+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c031400 con 0x7f0d88004f80 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.563+0000 7f0da3400640 1 -- 172.21.15.123:0/3764492701 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c009690 con 0x7f0d88004f80 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 -- 172.21.15.123:0/3764492701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88004f80 msgr2=0x7f0d8800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 --2- 172.21.15.123:0/3764492701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88004f80 0x7f0d8800dbd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009ac0 tx=0x7f0d88005830 comp rx=0 tx=0).stop 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9400640 1 -- 172.21.15.123:0/3764492701 reap_dead start 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 -- 172.21.15.123:0/3764492701 shutdown_connections 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 -- 172.21.15.123:0/3764492701 >> 172.21.15.123:0/3764492701 conn(0x7f0da40730d0 msgr2=0x7f0da40734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 -- 172.21.15.123:0/3764492701 shutdown_connections 2024-04-25T19:04:29.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.564+0000 7f0da9e00640 1 -- 172.21.15.123:0/3764492701 wait complete. 2024-04-25T19:04:29.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.566+0000 7f0da9e00640 1 Processor -- start 2024-04-25T19:04:29.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.567+0000 7f0da9e00640 1 -- start start 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d88004f80 0x7f0da4142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4159fa0 0x7f0da4144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40d0810 con 0x7f0da4133ba0 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40d0530 con 0x7f0da4159fa0 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da9e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40d0af0 con 0x7f0d88004f80 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41152/0 (socket says 172.21.15.123:41152) 2024-04-25T19:04:29.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.568+0000 7f0da3e00640 1 -- 172.21.15.123:0/4237285709 learned_addr learned my addr 172.21.15.123:0/4237285709 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da8a00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d88004f80 0x7f0da4142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da9400640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4159fa0 0x7f0da4144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 -- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d88004f80 msgr2=0x7f0da4142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d88004f80 0x7f0da4142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 -- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4159fa0 msgr2=0x7f0da4144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4159fa0 0x7f0da4144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 -- 172.21.15.123:0/4237285709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da415a3b0 con 0x7f0da4133ba0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.569+0000 7f0da3e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e870 tx=0x7f0d8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.570+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00c8d0 con 0x7f0da4133ba0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.570+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c018070 con 0x7f0da4133ba0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.570+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00ccd0 con 0x7f0da4133ba0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.570+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da41543b0 con 0x7f0da4133ba0 2024-04-25T19:04:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.570+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da4156550 con 0x7f0da4133ba0 2024-04-25T19:04:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.571+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da41562c0 con 0x7f0da4133ba0 2024-04-25T19:04:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.572+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d8c01d460 con 0x7f0da4133ba0 2024-04-25T19:04:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.573+0000 7f0da0c00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 0x7f0d800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.573+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0d8c0953c0 con 0x7f0da4133ba0 2024-04-25T19:04:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.574+0000 7f0da8a00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 0x7f0d800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.574+0000 7f0da8a00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 0x7f0d800769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009980 tx=0x7f0d9400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.578+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0da415a3b0 con 0x7f0da4133ba0 2024-04-25T19:04:29.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.846+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0da4050740 con 0x7f0da4133ba0 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.853+0000 7f0da0c00640 1 -- 172.21.15.123:0/4237285709 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f0d8c05f9c0 con 0x7f0da4133ba0 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHqCpmMqUvNhAA5Rf0ooG9ZwRKfPr3VFwX6A== 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stdout: caps mds = "allow" 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stdout: caps mon = "allow r" 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T19:04:29.855 INFO:teuthology.orchestra.run.smithi123.stderr:updated caps for client.0 2024-04-25T19:04:29.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.855+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 msgr2=0x7f0d800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.855+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 0x7f0d800769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009980 tx=0x7f0d9400b040 comp rx=0 tx=0).stop 2024-04-25T19:04:29.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.855+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 msgr2=0x7f0da4159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.855+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e870 tx=0x7f0d8c00ed30 comp rx=0 tx=0).stop 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 shutdown_connections 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0d800744e0 0x7f0d800769f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4159fa0 0x7f0da4144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4133ba0 0x7f0da4159a60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 --2- 172.21.15.123:0/4237285709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d88004f80 0x7f0da4142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 >> 172.21.15.123:0/4237285709 conn(0x7f0da40730d0 msgr2=0x7f0da415c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 shutdown_connections 2024-04-25T19:04:29.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:29.856+0000 7f0da9e00640 1 -- 172.21.15.123:0/4237285709 wait complete. 2024-04-25T19:04:29.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:04:30.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.147+0000 7f58cf600640 1 Processor -- start 2024-04-25T19:04:30.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.147+0000 7f58cf600640 1 -- start start 2024-04-25T19:04:30.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cf600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f58c80b57f0 0x7f58c80b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:30.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cf600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c80b6140 con 0x7f58c80ae7e0 2024-04-25T19:04:30.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cf600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c80b4750 con 0x7f58c815df00 2024-04-25T19:04:30.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cf600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f58c815e310 con 0x7f58c80b57f0 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58c7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f58c80b57f0 0x7f58c80b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cd600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f58c80ae7e0 0x7f58c80aebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36444/0 (socket says 172.21.15.123:36444) 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.150+0000 7f58cd600640 1 -- 172.21.15.123:0/2934939139 learned_addr learned my addr 172.21.15.123:0/2934939139 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.151+0000 7f58c7e00640 1 -- 172.21.15.123:0/2934939139 >> v1:172.21.15.146:6789/0 conn(0x7f58c815df00 legacy=0x7f58c806f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.151+0000 7f58c7e00640 1 -- 172.21.15.123:0/2934939139 >> v1:172.21.15.123:6789/0 conn(0x7f58c80ae7e0 legacy=0x7f58c80aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.151+0000 7f58c7e00640 1 -- 172.21.15.123:0/2934939139 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80b4a30 con 0x7f58c80b57f0 2024-04-25T19:04:30.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.152+0000 7f58c7e00640 1 --2- 172.21.15.123:0/2934939139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80b57f0 0x7f58c80b5c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58b40097a0 tx=0x7f58b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2148f797f5f697a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:30.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.152+0000 7f58c7400640 1 -- 172.21.15.123:0/2934939139 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4037070 con 0x7f58c80b57f0 2024-04-25T19:04:30.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.152+0000 7f58c7400640 1 -- 172.21.15.123:0/2934939139 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b402ae50 con 0x7f58c80b57f0 2024-04-25T19:04:30.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.152+0000 7f58c7400640 1 -- 172.21.15.123:0/2934939139 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4032660 con 0x7f58c80b57f0 2024-04-25T19:04:30.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 -- 172.21.15.123:0/2934939139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80b57f0 msgr2=0x7f58c80b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 --2- 172.21.15.123:0/2934939139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80b57f0 0x7f58c80b5c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58b40097a0 tx=0x7f58b402a140 comp rx=0 tx=0).stop 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 -- 172.21.15.123:0/2934939139 shutdown_connections 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 --2- 172.21.15.123:0/2934939139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80b57f0 0x7f58c80b5c00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 -- 172.21.15.123:0/2934939139 >> 172.21.15.123:0/2934939139 conn(0x7f58c80d1ae0 msgr2=0x7f58c80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 -- 172.21.15.123:0/2934939139 shutdown_connections 2024-04-25T19:04:30.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.153+0000 7f58cf600640 1 -- 172.21.15.123:0/2934939139 wait complete. 2024-04-25T19:04:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.155+0000 7f58cf600640 1 Processor -- start 2024-04-25T19:04:30.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.155+0000 7f58cf600640 1 -- start start 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 0x7f58c80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c815df00 0x7f58c80d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80b4ff0 con 0x7f58c80ae7e0 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80b5360 con 0x7f58c815df00 2024-04-25T19:04:30.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58cf600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58c806f950 con 0x7f58c80b57f0 2024-04-25T19:04:30.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.157+0000 7f58ccc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:30.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41160/0 (socket says 172.21.15.123:41160) 2024-04-25T19:04:30.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 -- 172.21.15.123:0/1568110068 learned_addr learned my addr 172.21.15.123:0/1568110068 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:30.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58c7e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 0x7f58c80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58cd600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c815df00 0x7f58c80d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 -- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 msgr2=0x7f58c80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 0x7f58c80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 -- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c815df00 msgr2=0x7f58c80d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c815df00 0x7f58c80d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 -- 172.21.15.123:0/1568110068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80cb260 con 0x7f58c80ae7e0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58c7e00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 0x7f58c80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.158+0000 7f58ccc00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00b3f0 tx=0x7f58bc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.159+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc015020 con 0x7f58c80ae7e0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.159+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc011830 con 0x7f58c80ae7e0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.159+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011c20 con 0x7f58c80ae7e0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.159+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c80cb540 con 0x7f58c80ae7e0 2024-04-25T19:04:30.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.159+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c80cbb00 con 0x7f58c80ae7e0 2024-04-25T19:04:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.160+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c80b30d0 con 0x7f58c80ae7e0 2024-04-25T19:04:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.161+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f58bc018020 con 0x7f58c80ae7e0 2024-04-25T19:04:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.163+0000 7f58c4c00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 0x7f58a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:30.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.163+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f58bc095060 con 0x7f58c80ae7e0 2024-04-25T19:04:30.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.163+0000 7f58c7e00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 0x7f58a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:30.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.164+0000 7f58c7e00640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 0x7f58a80769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f58b4034000 tx=0x7f58b402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.168+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58c80b30d0 con 0x7f58c80ae7e0 2024-04-25T19:04:30.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.411+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c8076a40 con 0x7f58c80ae7e0 2024-04-25T19:04:30.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.412+0000 7f58c4c00640 1 -- 172.21.15.123:0/1568110068 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7f58bc01f030 con 0x7f58c80ae7e0 2024-04-25T19:04:30.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:30.414 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:04:30.415 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 msgr2=0x7f58a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 0x7f58a80769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f58b4034000 tx=0x7f58b402e000 comp rx=0 tx=0).stop 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 msgr2=0x7f58c80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00b3f0 tx=0x7f58bc00b8b0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 shutdown_connections 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f58a80744e0 0x7f58a80769f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c815df00 0x7f58c80d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80b57f0 0x7f58c80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 --2- 172.21.15.123:0/1568110068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c80ae7e0 0x7f58c80ce740 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:30.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 >> 172.21.15.123:0/1568110068 conn(0x7f58c80d1ae0 msgr2=0x7f58c80739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:30.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 shutdown_connections 2024-04-25T19:04:30.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:30.415+0000 7f58cf600640 1 -- 172.21.15.123:0/1568110068 wait complete. 2024-04-25T19:04:30.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T19:04:23.546022+0000', 'modified': '2024-04-25T19:04:26.642728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4801': {'gid': 4801, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6835/4124573423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 4124573423}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 4124573423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4801, 'qdb_cluster': [4801]} 2024-04-25T19:04:30.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T19:04:30.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T19:04:31.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.133+0000 7ff5ff200640 1 Processor -- start 2024-04-25T19:04:31.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.133+0000 7ff5ff200640 1 -- start start 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5ff200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5ff200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5f815e2f0 0x7ff5f815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5ff200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f806ded0 con 0x7ff5f806fe20 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5ff200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f815f070 con 0x7ff5f815e2f0 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5ff200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f806f340 con 0x7ff5f806d580 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5fde00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff5f806fe20 0x7ff5f815b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36456/0 (socket says 172.21.15.123:36456) 2024-04-25T19:04:31.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.135+0000 7ff5fde00640 1 -- 172.21.15.123:0/1043928921 learned_addr learned my addr 172.21.15.123:0/1043928921 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093535874 0 0) 0x7ff5ec001110 con 0x7ff5f806fe20 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5e0003520 con 0x7ff5f806fe20 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fe800640 1 --2- 172.21.15.123:0/1043928921 >> v2:172.21.15.146:3300/0 conn(0x7ff5f815e2f0 0x7ff5f815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 396482945 0 0) 0x7ff5e0003520 con 0x7ff5f806fe20 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 msgr2=0x7ff5f806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5f806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 >> v2:172.21.15.146:3300/0 conn(0x7ff5f815e2f0 msgr2=0x7ff5f815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 --2- 172.21.15.123:0/1043928921 >> v2:172.21.15.146:3300/0 conn(0x7ff5f815e2f0 0x7ff5f815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f806f620 con 0x7ff5f806fe20 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7ff5ec001840 con 0x7ff5f806fe20 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 >> v1:172.21.15.123:6789/0 conn(0x7ff5f806fe20 legacy=0x7ff5f815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e0004d40 0x7ff5e0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f815f070 con 0x7ff5e00039b0 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f806f340 con 0x7ff5e0004d40 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.136+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e000bc80 con 0x7ff5f806d580 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fe800640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e0004d40 0x7ff5e0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 -- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 msgr2=0x7ff5e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 -- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e0004d40 msgr2=0x7ff5e0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5e0004d40 0x7ff5e0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fde00640 1 -- 172.21.15.123:0/1043928921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e0003520 con 0x7ff5e00039b0 2024-04-25T19:04:31.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.137+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f806d580 0x7ff5e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.138+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5e0003e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8070d80 tx=0x7ff5e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca761ccb76e3d57d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.138+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec036070 con 0x7ff5e00039b0 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.138+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec007e50 con 0x7ff5e00039b0 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.138+0000 7ff5fca00640 1 -- 172.21.15.123:0/1043928921 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec005d50 con 0x7ff5e00039b0 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 -- 172.21.15.123:0/1043928921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 msgr2=0x7ff5e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 --2- 172.21.15.123:0/1043928921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5e0003e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8070d80 tx=0x7ff5e0005830 comp rx=0 tx=0).stop 2024-04-25T19:04:31.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5fe800640 1 -- 172.21.15.123:0/1043928921 reap_dead start 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 -- 172.21.15.123:0/1043928921 shutdown_connections 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 -- 172.21.15.123:0/1043928921 >> 172.21.15.123:0/1043928921 conn(0x7ff5f8073200 msgr2=0x7ff5f8073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 -- 172.21.15.123:0/1043928921 shutdown_connections 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.139+0000 7ff5ff200640 1 -- 172.21.15.123:0/1043928921 wait complete. 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.141+0000 7ff5ff200640 1 Processor -- start 2024-04-25T19:04:31.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.142+0000 7ff5ff200640 1 -- start start 2024-04-25T19:04:31.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f81e7a50 0x7ff5f80b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 0x7ff5f80b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f815edc0 con 0x7ff5e00039b0 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f815ec40 con 0x7ff5f8132860 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5ff200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f806d990 con 0x7ff5f81e7a50 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41176/0 (socket says 172.21.15.123:41176) 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.143+0000 7ff5fde00640 1 -- 172.21.15.123:0/2458579237 learned_addr learned my addr 172.21.15.123:0/2458579237 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fd400640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f81e7a50 0x7ff5f80b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 -- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f81e7a50 msgr2=0x7ff5f80b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f81e7a50 0x7ff5f80b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fe800640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 0x7ff5f80b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 -- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 msgr2=0x7ff5f80b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 0x7ff5f80b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 -- 172.21.15.123:0/2458579237 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f80b4740 con 0x7ff5e00039b0 2024-04-25T19:04:31.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fe800640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 0x7ff5f80b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:31.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.144+0000 7ff5fde00640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02af00 tx=0x7ff5f806f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.145+0000 7ff5f2000640 1 -- 172.21.15.123:0/2458579237 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec036040 con 0x7ff5e00039b0 2024-04-25T19:04:31.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.145+0000 7ff5f2000640 1 -- 172.21.15.123:0/2458579237 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec005bb0 con 0x7ff5e00039b0 2024-04-25T19:04:31.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.145+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f80b4a20 con 0x7ff5e00039b0 2024-04-25T19:04:31.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.145+0000 7ff5f2000640 1 -- 172.21.15.123:0/2458579237 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec007070 con 0x7ff5e00039b0 2024-04-25T19:04:31.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.145+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f80b4fe0 con 0x7ff5e00039b0 2024-04-25T19:04:31.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.147+0000 7ff5f2000640 1 -- 172.21.15.123:0/2458579237 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5ec007210 con 0x7ff5e00039b0 2024-04-25T19:04:31.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.148+0000 7ff5f2000640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 0x7ff5dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.148+0000 7ff5f2000640 1 -- 172.21.15.123:0/2458579237 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff5ec0b56e0 con 0x7ff5e00039b0 2024-04-25T19:04:31.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.148+0000 7ff5fd400640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 0x7ff5dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.149+0000 7ff5fd400640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 0x7ff5dc0769f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5f81604e0 tx=0x7ff5e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.149+0000 7ff607864740 1 Processor -- start 2024-04-25T19:04:31.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.149+0000 7ff607864740 1 -- start start 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdd8270 0x5558cc1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558cc0de600 con 0x5558cbdb7590 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5558cc07b100 con 0x5558cbdd8270 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5558cc0da5c0 con 0x5558cbdaf3b0 2024-04-25T19:04:31.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff5fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff5fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41184/0 (socket says 172.21.15.123:41184) 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff5fe800640 1 -- 172.21.15.123:0/4248485151 learned_addr learned my addr 172.21.15.123:0/4248485151 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff5fd400640 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.151+0000 7ff5fde00640 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdd8270 0x5558cc1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 -- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 msgr2=0x5558cc1bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 -- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdd8270 msgr2=0x5558cc1a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdd8270 0x5558cc1a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 -- 172.21.15.123:0/4248485151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558cbdf02d0 con 0x5558cbdb7590 2024-04-25T19:04:31.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5fe800640 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0004780 tx=0x7ff5f4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4aec1c8e7bd98e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5e6600640 1 -- 172.21.15.123:0/4248485151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4009aa0 con 0x5558cbdb7590 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5e6600640 1 -- 172.21.15.123:0/4248485151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x5558cbdb7590 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.152+0000 7ff5e6600640 1 -- 172.21.15.123:0/4248485151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4015420 con 0x5558cbdb7590 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 -- 172.21.15.123:0/4248485151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 msgr2=0x5558cc1c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0004780 tx=0x7ff5f4004410 comp rx=0 tx=0).stop 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 -- 172.21.15.123:0/4248485151 shutdown_connections 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1c0480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 --2- 172.21.15.123:0/4248485151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdd8270 0x5558cc1a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 -- 172.21.15.123:0/4248485151 >> 172.21.15.123:0/4248485151 conn(0x5558cbdcc3c0 msgr2=0x5558cc17a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 -- 172.21.15.123:0/4248485151 shutdown_connections 2024-04-25T19:04:31.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.153+0000 7ff607864740 1 -- 172.21.15.123:0/4248485151 wait complete. 2024-04-25T19:04:31.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.154+0000 7ff607864740 1 -- start start 2024-04-25T19:04:31.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.155+0000 7ff607864740 1 Processor -- start 2024-04-25T19:04:31.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdaf3b0 0x5558cc1bbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc1cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558cc065f70 con 0x5558cbdb7590 2024-04-25T19:04:31.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5558cc1bb000 con 0x5558cbdaf3b0 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5558cc1bb2e0 con 0x5558cbdd8270 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.156+0000 7ff607864740 10 client.0 _ll_register_callbacks cb 0x5558cc0d4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41198/0 (socket says 172.21.15.123:41198) 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 -- 172.21.15.123:0/1766852276 learned_addr learned my addr 172.21.15.123:0/1766852276 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdaf3b0 0x5558cc1bbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fe800640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc1cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 -- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 msgr2=0x5558cc1cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc1cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 -- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdaf3b0 msgr2=0x5558cc1bbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdaf3b0 0x5558cc1bbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 -- 172.21.15.123:0/1766852276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558cbeba6b0 con 0x5558cbdb7590 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fe800640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc1cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.157+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800fab0 tx=0x7ff5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f670d5dcfa15909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:04:31.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8028060 con 0x5558cbdb7590 2024-04-25T19:04:31.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff5e8003da0 con 0x5558cbdb7590 2024-04-25T19:04:31.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f401a070 con 0x5558cbdb7590 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff5e802e050 con 0x5558cbdb7590 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff607864740 10 client.5499 fetch_fsmap learned FSMap version 18 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5558cc1d0090 con 0x5558cbdb7590 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x5558cbdb7590 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.158+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7ff5e8031020 con 0x5558cbdb7590 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 10 client.5499 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 10 client.5499 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 20 client.5499 populate_metadata read hostname 'smithi123' 2024-04-25T19:04:31.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cbd96e40 0x5558cc1dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 4 client.5499 mds_command: new command op to 4801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 --> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558cc0ec210 con 0x5558cbd96e40 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cbd96e40 0x5558cc1dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cbd96e40 0x5558cc1dd4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6807bdc5885a9e8b server_cookie=534684a428439f6c in_seq=0 out_seq=0 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.159+0000 7ff5daa00640 10 client.5499 ms_handle_connect on v2:172.21.15.123:6834/4124573423 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.160+0000 7ff5daa00640 1 -- 172.21.15.123:0/1766852276 <== mds.0 v2:172.21.15.123:6834/4124573423 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558cc0ec210 con 0x5558cbd96e40 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.160+0000 7ff5daa00640 10 client.5499 handle_command_reply: tid=0 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.160+0000 7ff607864740 1 client.5499 shutdown 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 msgr2=0x5558cc1bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800fab0 tx=0x7ff5e80046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 shutdown_connections 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cbd96e40 0x5558cc1dd4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc1cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdb7590 0x5558cc1bf7e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 --2- 172.21.15.123:0/1766852276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdaf3b0 0x5558cc1bbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff5daa00640 0 client.5499 ms_handle_reset on v2:172.21.15.123:6834/4124573423 2024-04-25T19:04:31.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 >> 172.21.15.123:0/1766852276 conn(0x5558cbdcc3c0 msgr2=0x5558cc17a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 shutdown_connections 2024-04-25T19:04:31.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 1 -- 172.21.15.123:0/1766852276 wait complete. 2024-04-25T19:04:31.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.161+0000 7ff607864740 20 client.5499 trim_cache size 0 max 16384 2024-04-25T19:04:31.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.182+0000 7ff607864740 1 Processor -- start 2024-04-25T19:04:31.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.182+0000 7ff607864740 1 -- start start 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc19e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f82029d0 con 0x5558cbdaf3b0 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8201d60 con 0x5558cbdb7590 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f8202380 con 0x5558cbdd8270 2024-04-25T19:04:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff5fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41214/0 (socket says 172.21.15.123:41214) 2024-04-25T19:04:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.183+0000 7ff5fde00640 1 -- 172.21.15.123:0/1550385604 learned_addr learned my addr 172.21.15.123:0/1550385604 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fe800640 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fd400640 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 -- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 msgr2=0x5558cc19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc19e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 -- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 msgr2=0x5558cc1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 -- 172.21.15.123:0/1550385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558cc1bef00 con 0x5558cbdaf3b0 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5fde00640 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec07f4d0 tx=0x7ff5ec0b38b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fd7f05ab48455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5e5c00640 1 -- 172.21.15.123:0/1550385604 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0838f0 con 0x5558cbdaf3b0 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5e5c00640 1 -- 172.21.15.123:0/1550385604 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec007e50 con 0x5558cbdaf3b0 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.184+0000 7ff5e5c00640 1 -- 172.21.15.123:0/1550385604 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0883a0 con 0x5558cbdaf3b0 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 -- 172.21.15.123:0/1550385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 msgr2=0x5558cc1a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec07f4d0 tx=0x7ff5ec0b38b0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 -- 172.21.15.123:0/1550385604 shutdown_connections 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdd8270 0x5558cc19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 --2- 172.21.15.123:0/1550385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdaf3b0 0x5558cc1a5650 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.185+0000 7ff607864740 1 -- 172.21.15.123:0/1550385604 >> 172.21.15.123:0/1550385604 conn(0x5558cbdcc3c0 msgr2=0x5558cc1bb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.186+0000 7ff607864740 1 -- 172.21.15.123:0/1550385604 shutdown_connections 2024-04-25T19:04:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.186+0000 7ff607864740 1 -- 172.21.15.123:0/1550385604 wait complete. 2024-04-25T19:04:31.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.186+0000 7ff607864740 1 -- start start 2024-04-25T19:04:31.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.187+0000 7ff607864740 1 Processor -- start 2024-04-25T19:04:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc17afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558cc1bd060 con 0x5558cbdd8270 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5558cc1bcc10 con 0x5558cbdb7590 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5558cc1bd680 con 0x5558cbdaf3b0 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.188+0000 7ff607864740 10 client.0 _ll_register_callbacks cb 0x7ff5e8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41218/0 (socket says 172.21.15.123:41218) 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 -- 172.21.15.123:0/3275562505 learned_addr learned my addr 172.21.15.123:0/3275562505 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fde00640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 -- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 msgr2=0x5558cc1bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 -- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 msgr2=0x5558cc17afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fd400640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc17afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc17afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 -- 172.21.15.123:0/3275562505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558cc175860 con 0x5558cbdd8270 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5fe800640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4004450 tx=0x7ff5f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7155736b7ced1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.189+0000 7ff5daa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4009dd0 con 0x5558cbdd8270 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec007e50 con 0x5558cbdd8270 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff5f4029c70 con 0x5558cbdd8270 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff5f4015420 con 0x5558cbdd8270 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 10 client.5505 fetch_fsmap learned FSMap version 18 2024-04-25T19:04:31.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5558cc1d7520 con 0x5558cbdd8270 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec007e50 con 0x5558cbdd8270 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7ff5f40155c0 con 0x5558cbdd8270 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 10 client.5505 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 10 client.5505 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 20 client.5505 populate_metadata read hostname 'smithi123' 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cc19ccf0 0x5558cc19d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 4 client.5505 mds_command: new command op to 4801 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.190+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 --> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] -- command(tid 0: {"prefix": "status"}) -- 0x5558cc0e6080 con 0x5558cc19ccf0 2024-04-25T19:04:31.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.191+0000 7ff5fde00640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cc19ccf0 0x5558cc19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.191+0000 7ff5fde00640 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cc19ccf0 0x5558cc19d100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5cc2bd2a78a89e server_cookie=c97c5b71e0c1241e in_seq=0 out_seq=0 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.191+0000 7ff5daa00640 10 client.5505 ms_handle_connect on v2:172.21.15.123:6834/4124573423 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff5daa00640 1 -- 172.21.15.123:0/3275562505 <== mds.0 v2:172.21.15.123:6834/4124573423 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5558cc0e6080 con 0x5558cc19ccf0 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff5daa00640 10 client.5505 handle_command_reply: tid=0 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 client.5505 shutdown 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 msgr2=0x5558cc17b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4004450 tx=0x7ff5f4004a40 comp rx=0 tx=0).stop 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 shutdown_connections 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:6834/4124573423,v1:172.21.15.123:6835/4124573423] conn(0x5558cc19ccf0 0x5558cc19d100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558cbdd8270 0x5558cc17b4f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5558cbdb7590 0x5558cc17afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 --2- 172.21.15.123:0/3275562505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5558cbdaf3b0 0x5558cc1bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff5daa00640 0 client.5505 ms_handle_reset on v2:172.21.15.123:6834/4124573423 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 >> 172.21.15.123:0/3275562505 conn(0x5558cbdcc3c0 msgr2=0x5558cc19e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 shutdown_connections 2024-04-25T19:04:31.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 1 -- 172.21.15.123:0/3275562505 wait complete. 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.192+0000 7ff607864740 20 client.5505 trim_cache size 0 max 16384 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "a2d72801-e932-4284-9776-fce727cd5af1", 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4801, 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 7.1425975660000001, 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 18, 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 40, 2024-04-25T19:04:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 40, 2024-04-25T19:04:31.198 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 16.464869649000001 2024-04-25T19:04:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:04:31.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 msgr2=0x7ff5dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 0x7ff5dc0769f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5f81604e0 tx=0x7ff5e8008040 comp rx=0 tx=0).stop 2024-04-25T19:04:31.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 msgr2=0x7ff5f80cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02af00 tx=0x7ff5f806f7f0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 shutdown_connections 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff5dc0744e0 0x7ff5dc0769f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5f8132860 0x7ff5f80b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5f81e7a50 0x7ff5f80b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 --2- 172.21.15.123:0/2458579237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e00039b0 0x7ff5f80cb1b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 >> 172.21.15.123:0/2458579237 conn(0x7ff5f8073200 msgr2=0x7ff5f806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 shutdown_connections 2024-04-25T19:04:31.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.193+0000 7ff5ff200640 1 -- 172.21.15.123:0/2458579237 wait complete. 2024-04-25T19:04:31.218 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:04:31.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.486+0000 7f4b4ae00640 1 Processor -- start 2024-04-25T19:04:31.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.486+0000 7f4b4ae00640 1 -- start start 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b4ae00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b4415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b4ae00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4b440709f0 0x7f4b4406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b4ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b44168440 con 0x7f4b44165360 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b4ae00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4b44168720 con 0x7f4b440709f0 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b4ae00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b44168a00 con 0x7f4b44167ed0 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b49000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4b44165360 0x7f4b440702e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36470/0 (socket says 172.21.15.123:36470) 2024-04-25T19:04:31.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.489+0000 7f4b49000640 1 -- 172.21.15.123:0/3555176157 learned_addr learned my addr 172.21.15.123:0/3555176157 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b49a00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b4415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b4a400640 1 --2- 172.21.15.123:0/3555176157 >> v2:172.21.15.146:3300/0 conn(0x7f4b440709f0 0x7f4b4406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390742432 0 0) 0x7f4b44168440 con 0x7f4b44165360 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b2c003520 con 0x7f4b44165360 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3336825297 0 0) 0x7f4b2c003520 con 0x7f4b44165360 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 msgr2=0x7f4b4415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b4415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 >> v2:172.21.15.146:3300/0 conn(0x7f4b440709f0 msgr2=0x7f4b4406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 --2- 172.21.15.123:0/3555176157 >> v2:172.21.15.146:3300/0 conn(0x7f4b440709f0 0x7f4b4406ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b44168ce0 con 0x7f4b44165360 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f4b34001690 con 0x7f4b44165360 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 >> v1:172.21.15.123:6789/0 conn(0x7f4b44165360 legacy=0x7f4b440702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c0039b0 0x7f4b2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2c004d40 0x7f4b2c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44168720 con 0x7f4b2c004d40 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44168a00 con 0x7f4b2c0039b0 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.490+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c00b9c0 con 0x7f4b44167ed0 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2c004d40 0x7f4b2c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b49a00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c0039b0 0x7f4b2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b49000640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 -- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 msgr2=0x7f4b2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b44167ed0 0x7f4b2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 -- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c0039b0 msgr2=0x7f4b2c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:31.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c0039b0 0x7f4b2c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.491+0000 7f4b4a400640 1 -- 172.21.15.123:0/3555176157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c003520 con 0x7f4b2c004d40 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.492+0000 7f4b4a400640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2c004d40 0x7f4b2c005150 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009fc0 tx=0x7f4b2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b401d1e1f6f01729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.493+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c009c10 con 0x7f4b2c004d40 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.493+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c036070 con 0x7f4b2c004d40 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.493+0000 7f4b43e00640 1 -- 172.21.15.123:0/3555176157 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c02d070 con 0x7f4b2c004d40 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 -- 172.21.15.123:0/3555176157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2c004d40 msgr2=0x7f4b2c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/3555176157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2c004d40 0x7f4b2c005150 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009fc0 tx=0x7f4b2c005830 comp rx=0 tx=0).stop 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4a400640 1 -- 172.21.15.123:0/3555176157 reap_dead start 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 -- 172.21.15.123:0/3555176157 shutdown_connections 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 -- 172.21.15.123:0/3555176157 >> 172.21.15.123:0/3555176157 conn(0x7f4b440ce090 msgr2=0x7f4b440ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 -- 172.21.15.123:0/3555176157 shutdown_connections 2024-04-25T19:04:31.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.494+0000 7f4b4ae00640 1 -- 172.21.15.123:0/3555176157 wait complete. 2024-04-25T19:04:31.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.496+0000 7f4b4ae00640 1 Processor -- start 2024-04-25T19:04:31.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.496+0000 7f4b4ae00640 1 -- start start 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 0x7f4b4406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4406d090 0x7f4b4415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4415ea30 con 0x7f4b440c69e0 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44165770 con 0x7f4b4406d090 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b4ae00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4415ebb0 con 0x7f4b441626c0 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b49a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b49a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41242/0 (socket says 172.21.15.123:41242) 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.498+0000 7f4b49a00640 1 -- 172.21.15.123:0/2702450320 learned_addr learned my addr 172.21.15.123:0/2702450320 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49000640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 0x7f4b4406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b4a400640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4406d090 0x7f4b4415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 -- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 msgr2=0x7f4b4406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 0x7f4b4406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 -- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4406d090 msgr2=0x7f4b4415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4406d090 0x7f4b4415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 -- 172.21.15.123:0/2702450320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b440c6df0 con 0x7f4b440c69e0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49000640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 0x7f4b4406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b49a00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4b38011590 tx=0x7f4b38011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38019070 con 0x7f4b440c69e0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38003610 con 0x7f4b440c69e0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b380039e0 con 0x7f4b440c69e0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b44162df0 con 0x7f4b440c69e0 2024-04-25T19:04:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.499+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441633b0 con 0x7f4b440c69e0 2024-04-25T19:04:31.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.501+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b44050740 con 0x7f4b440c69e0 2024-04-25T19:04:31.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.502+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b3801c070 con 0x7f4b440c69e0 2024-04-25T19:04:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.504+0000 7f4b41600640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 0x7f4b280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.504+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4b380957a0 con 0x7f4b440c69e0 2024-04-25T19:04:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.505+0000 7f4b49000640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 0x7f4b280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.505+0000 7f4b49000640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 0x7f4b280769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001800 tx=0x7f4b34001c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:31.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.509+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b3805fd80 con 0x7f4b440c69e0 2024-04-25T19:04:31.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.748+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4b44073da0 con 0x7f4b440c69e0 2024-04-25T19:04:31.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.749+0000 7f4b41600640 1 -- 172.21.15.123:0/2702450320 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f4b3805f680 con 0x7f4b440c69e0 2024-04-25T19:04:31.751 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.751+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 msgr2=0x7f4b280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 0x7f4b280769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001800 tx=0x7f4b34001c90 comp rx=0 tx=0).stop 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 msgr2=0x7f4b44163e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4b38011590 tx=0x7f4b38011a50 comp rx=0 tx=0).stop 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 shutdown_connections 2024-04-25T19:04:31.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4b280744b0 0x7f4b280769c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4406d090 0x7f4b4415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b441626c0 0x7f4b4406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 --2- 172.21.15.123:0/2702450320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b440c69e0 0x7f4b44163e90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 >> 172.21.15.123:0/2702450320 conn(0x7f4b440ce090 msgr2=0x7f4b4414d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 shutdown_connections 2024-04-25T19:04:31.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:31.752+0000 7f4b4ae00640 1 -- 172.21.15.123:0/2702450320 wait complete. 2024-04-25T19:04:31.779 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T19:04:31.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:04:31.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-25T19:04:31.779 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T19:04:31.780 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:04:31.780 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T19:04:31.780 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T19:04:31.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:04:31.780 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:04:31.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:31.780 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-25T19:04:31.827 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:04:31.827 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:04:31.827 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:04:31.827 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.827 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42525sec preferred_lft 42525sec 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.828 INFO:teuthology.orchestra.run.smithi123.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 62:a3:19:86:2b:d1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::942b:40ff:fe0f:8567/64 scope link 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 62:a3:19:86:2b:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::60a3:19ff:fe86:2bd1/64 scope link 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether c2:c9:92:b3:0f:67 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::c0c9:92ff:feb3:f67/64 scope link 2024-04-25T19:04:31.829 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:04:31.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:31.831 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:04:31.901 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-25T19:04:31.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:31.905 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-25T19:04:31.957 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-25T19:04:31.957 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:04:31.957 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:04:31.957 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:04:31.958 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:04:31.958 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-25T19:04:32.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:04:32.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:04:32.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:32.163 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:04:32.208 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:04:32.209 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T19:04:32.209 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:04:32.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:32.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-25T19:04:32.337 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:04:32.337 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T19:04:32.389 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:04:32.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:04:32.389 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:04:32.439 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T19:04:32.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:32.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:04:32.645 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:04:32.646 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:04:32.687 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T19:04:32.687 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T19:04:32.687 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T19:04:32.687 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 0@a2d72801-e932-4284-9776-fce727cd5af1.cephfs=/ 2024-04-25T19:04:32.687 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.123:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T19:04:32.689 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T19:04:32.689 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:04:32.689 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:04:32.690 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:04:32.690 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:04:32.694 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:04:32.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:32.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:04:32.773 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:04:32.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:04:32.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:04:32.902 INFO:teuthology.orchestra.run.smithi123.stdout:client4739 2024-04-25T19:04:32.902 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:04:32.902 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:04:32.950 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=xfstests-dev 2024-04-25T19:04:33.047 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.s4sluTBO7Yxfstests-dev 2024-04-25T19:04:33.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:33.047 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.s4sluTBO7Yxfstests-dev) 2024-04-25T19:04:33.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:33.132 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.s4sluTBO7Yxfstests-dev) 2024-04-25T19:04:33.191 INFO:teuthology.orchestra.run.smithi123.stderr:Cloning into '/tmp/tmp.s4sluTBO7Yxfstests-dev'... 2024-04-25T19:04:34.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:34.176 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-25T19:04:34.242 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=test 2024-04-25T19:04:34.296 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.iUM1CQACHytest 2024-04-25T19:04:34.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:04:34.296 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-25T19:04:34.349 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=scratch 2024-04-25T19:04:34.399 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.L6JI355wCGscratch 2024-04-25T19:04:34.399 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-25T19:04:34.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-25T19:04:35.170 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:55 ago on Thu 25 Apr 2024 06:59:40 PM UTC. 2024-04-25T19:04:35.251 INFO:teuthology.orchestra.run.smithi123.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T19:04:35.252 INFO:teuthology.orchestra.run.smithi123.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T19:04:35.253 INFO:teuthology.orchestra.run.smithi123.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-25T19:04:35.253 INFO:teuthology.orchestra.run.smithi123.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-25T19:04:35.254 INFO:teuthology.orchestra.run.smithi123.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-25T19:04:35.256 INFO:teuthology.orchestra.run.smithi123.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-04-25T19:04:35.258 INFO:teuthology.orchestra.run.smithi123.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-25T19:04:35.259 INFO:teuthology.orchestra.run.smithi123.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-04-25T19:04:35.260 INFO:teuthology.orchestra.run.smithi123.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-04-25T19:04:35.262 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T19:04:35.263 INFO:teuthology.orchestra.run.smithi123.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-25T19:04:35.265 INFO:teuthology.orchestra.run.smithi123.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-25T19:04:35.265 INFO:teuthology.orchestra.run.smithi123.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T19:04:35.266 INFO:teuthology.orchestra.run.smithi123.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-04-25T19:04:35.267 INFO:teuthology.orchestra.run.smithi123.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-04-25T19:04:35.268 INFO:teuthology.orchestra.run.smithi123.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-25T19:04:35.269 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-04-25T19:04:35.270 INFO:teuthology.orchestra.run.smithi123.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-25T19:04:35.271 INFO:teuthology.orchestra.run.smithi123.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-25T19:04:35.273 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T19:04:35.318 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-devel x86_64 5.14.0-437.el9 appstream 16 M 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-04-25T19:04:35.320 INFO:teuthology.orchestra.run.smithi123.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-04-25T19:04:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout:Install 17 Packages 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 24 M 2024-04-25T19:04:35.322 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 78 M 2024-04-25T19:04:35.323 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-25T19:04:36.191 INFO:teuthology.orchestra.run.smithi123.stdout:(1/17): fio-engine-http-3.35-1.el9.x86_64.rpm 86 kB/s | 18 kB 00:00 2024-04-25T19:04:36.223 INFO:teuthology.orchestra.run.smithi123.stdout:(2/17): rmt-1.6-6.el9.x86_64.rpm 190 kB/s | 46 kB 00:00 2024-04-25T19:04:36.240 INFO:teuthology.orchestra.run.smithi123.stdout:(3/17): fio-engine-libaio-3.35-1.el9.x86_64.rpm 332 kB/s | 16 kB 00:00 2024-04-25T19:04:36.265 INFO:teuthology.orchestra.run.smithi123.stdout:(4/17): fio-engine-nbd-3.35-1.el9.x86_64.rpm 360 kB/s | 15 kB 00:00 2024-04-25T19:04:36.291 INFO:teuthology.orchestra.run.smithi123.stdout:(5/17): fio-engine-rados-3.35-1.el9.x86_64.rpm 320 kB/s | 16 kB 00:00 2024-04-25T19:04:36.349 INFO:teuthology.orchestra.run.smithi123.stdout:(6/17): fio-engine-rbd-3.35-1.el9.x86_64.rpm 198 kB/s | 16 kB 00:00 2024-04-25T19:04:36.383 INFO:teuthology.orchestra.run.smithi123.stdout:(7/17): fio-engine-rdma-3.35-1.el9.x86_64.rpm 226 kB/s | 21 kB 00:00 2024-04-25T19:04:36.583 INFO:teuthology.orchestra.run.smithi123.stdout:(8/17): libcap-devel-2.48-9.el9.x86_64.rpm 204 kB/s | 41 kB 00:00 2024-04-25T19:04:36.792 INFO:teuthology.orchestra.run.smithi123.stdout:(9/17): libnbd-1.18.1-3.el9.x86_64.rpm 851 kB/s | 166 kB 00:00 2024-04-25T19:04:37.126 INFO:teuthology.orchestra.run.smithi123.stdout:(10/17): kernel-devel-5.14.0-437.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T19:04:37.176 INFO:teuthology.orchestra.run.smithi123.stdout:(11/17): sqlite-3.34.1-7.el9.x86_64.rpm 1.9 MB/s | 748 kB 00:00 2024-04-25T19:04:37.201 INFO:teuthology.orchestra.run.smithi123.stdout:(12/17): udftools-2.2-5.el9.x86_64.rpm 2.5 MB/s | 149 kB 00:00 2024-04-25T19:04:37.235 INFO:teuthology.orchestra.run.smithi123.stdout:(13/17): gdbm-1.19-4.el9.x86_64.rpm 2.3 MB/s | 132 kB 00:00 2024-04-25T19:04:37.685 INFO:teuthology.orchestra.run.smithi123.stdout:(14/17): gdbm-devel-1.19-4.el9.x86_64.rpm 122 kB/s | 59 kB 00:00 2024-04-25T19:04:38.036 INFO:teuthology.orchestra.run.smithi123.stdout:(15/17): indent-2.2.13-5.el9.x86_64.rpm 515 kB/s | 179 kB 00:00 2024-04-25T19:04:38.319 INFO:teuthology.orchestra.run.smithi123.stdout:(16/17): dump-0.4-0.51.b47.el9.x86_64.rpm 81 kB/s | 88 kB 00:01 2024-04-25T19:04:38.544 INFO:teuthology.orchestra.run.smithi123.stdout:(17/17): fio-3.35-1.el9.x86_64.rpm 2.4 MB/s | 6.2 MB 00:02 2024-04-25T19:04:38.552 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:04:38.553 INFO:teuthology.orchestra.run.smithi123.stdout:Total 7.3 MB/s | 24 MB 00:03 2024-04-25T19:04:39.251 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-25T19:04:39.360 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-25T19:04:39.360 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-25T19:04:40.154 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-25T19:04:40.155 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-25T19:04:41.327 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-25T19:04:41.407 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 1/17 2024-04-25T19:04:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 2/17 2024-04-25T19:04:41.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-25T19:04:41.526 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-25T19:04:41.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-25T19:04:41.595 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-25T19:04:41.628 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-25T19:04:41.878 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-25T19:04:41.932 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-3.35-1.el9.x86_64 9/17 2024-04-25T19:04:41.972 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rmt-2:1.6-6.el9.x86_64 10/17 2024-04-25T19:04:42.027 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 11/17 2024-04-25T19:04:42.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 12/17 2024-04-25T19:04:42.180 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : indent-2.2.13-5.el9.x86_64 13/17 2024-04-25T19:04:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : udftools-2.2-5.el9.x86_64 14/17 2024-04-25T19:04:42.316 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 15/17 2024-04-25T19:04:53.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:53.259 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:53.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 16/17 2024-04-25T19:04:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-25T19:05:23.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:23.494 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-25T19:05:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/17 2024-04-25T19:05:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-3.35-1.el9.x86_64 2/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-devel-5.14.0-437.el9.x86_64 9/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 11/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/17 2024-04-25T19:05:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/17 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/17 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-25T19:05:44.602 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-437.el9.x86_64 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout: udftools-2.2-5.el9.x86_64 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:05:44.603 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-25T19:05:45.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd -m fsgqa 2024-04-25T19:05:45.083 INFO:teuthology.orchestra.run.smithi123.stderr:useradd: user 'fsgqa' already exists 2024-04-25T19:05:45.087 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T19:05:45.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo groupadd fsgqa 2024-04-25T19:05:45.146 INFO:teuthology.orchestra.run.smithi123.stderr:groupadd: group 'fsgqa' already exists 2024-04-25T19:05:45.149 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T19:05:45.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd fsgqa2 2024-04-25T19:05:45.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd 123456-fsgqa 2024-04-25T19:05:46.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-25T19:05:46.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.704+0000 7fee6b400640 1 Processor -- start 2024-04-25T19:05:46.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.704+0000 7fee6b400640 1 -- start start 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.706+0000 7fee6b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee6406d8f0 0x7fee64154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.706+0000 7fee6b400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee64071600 con 0x7fee64161b10 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.706+0000 7fee6b400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee640aec30 con 0x7fee640d1120 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.706+0000 7fee6b400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee640d1530 con 0x7fee6406d8f0 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee69400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee6406d8f0 0x7fee64154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee63e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fee64161b10 0x7fee64161f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56734/0 (socket says 172.21.15.123:56734) 2024-04-25T19:05:46.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee63e00640 1 -- 172.21.15.123:0/1668130411 learned_addr learned my addr 172.21.15.123:0/1668130411 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:05:46.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee69400640 1 -- 172.21.15.123:0/1668130411 >> v1:172.21.15.146:6789/0 conn(0x7fee640d1120 legacy=0x7fee64070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:46.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee69400640 1 -- 172.21.15.123:0/1668130411 >> v1:172.21.15.123:6789/0 conn(0x7fee64161b10 legacy=0x7fee64161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:46.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.707+0000 7fee69400640 1 -- 172.21.15.123:0/1668130411 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee640aef10 con 0x7fee6406d8f0 2024-04-25T19:05:46.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.708+0000 7fee69400640 1 --2- 172.21.15.123:0/1668130411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee64154950 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd729dddaa06d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:46.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.709+0000 7fee63400640 1 -- 172.21.15.123:0/1668130411 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007740 con 0x7fee6406d8f0 2024-04-25T19:05:46.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.709+0000 7fee63400640 1 -- 172.21.15.123:0/1668130411 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c0078e0 con 0x7fee6406d8f0 2024-04-25T19:05:46.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.709+0000 7fee63400640 1 -- 172.21.15.123:0/1668130411 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007cb0 con 0x7fee6406d8f0 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 -- 172.21.15.123:0/1668130411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 msgr2=0x7fee64154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 --2- 172.21.15.123:0/1668130411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee64154950 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02a590 comp rx=0 tx=0).stop 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 -- 172.21.15.123:0/1668130411 shutdown_connections 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 --2- 172.21.15.123:0/1668130411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee64154950 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 -- 172.21.15.123:0/1668130411 >> 172.21.15.123:0/1668130411 conn(0x7fee640d1ae0 msgr2=0x7fee640d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 -- 172.21.15.123:0/1668130411 shutdown_connections 2024-04-25T19:05:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.710+0000 7fee6b400640 1 -- 172.21.15.123:0/1668130411 wait complete. 2024-04-25T19:05:46.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.713+0000 7fee6b400640 1 Processor -- start 2024-04-25T19:05:46.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.713+0000 7fee6b400640 1 -- start start 2024-04-25T19:05:46.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:46.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee640d1120 0x7fee6409b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee64161b10 0x7fee6415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee640ae7e0 con 0x7fee6406d8f0 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee640af4d0 con 0x7fee64161b10 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.715+0000 7fee6b400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee640ae960 con 0x7fee640d1120 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34714/0 (socket says 172.21.15.123:34714) 2024-04-25T19:05:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 -- 172.21.15.123:0/1909030480 learned_addr learned my addr 172.21.15.123:0/1909030480 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:05:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 -- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee640d1120 msgr2=0x7fee6409b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee640d1120 0x7fee6409b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 -- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee64161b10 msgr2=0x7fee6415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee69400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee64161b10 0x7fee6415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee64161b10 0x7fee6415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.716+0000 7fee68a00640 1 -- 172.21.15.123:0/1909030480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641f4d60 con 0x7fee6406d8f0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee68a00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b3e0 tx=0x7fee5800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58015020 con 0x7fee6406d8f0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58011850 con 0x7fee6406d8f0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58011c40 con 0x7fee6406d8f0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6409c190 con 0x7fee6406d8f0 2024-04-25T19:05:46.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.717+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee64158bf0 con 0x7fee6406d8f0 2024-04-25T19:05:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.718+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64076a40 con 0x7fee6406d8f0 2024-04-25T19:05:46.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.720+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee58018020 con 0x7fee6406d8f0 2024-04-25T19:05:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.721+0000 7fee60c00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:46.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.721+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7fee58095390 con 0x7fee6406d8f0 2024-04-25T19:05:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.721+0000 7fee63e00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:46.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.722+0000 7fee63e00640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 0x7fee440769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee50000f60 tx=0x7fee50007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:46.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:46.726+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee5801f020 con 0x7fee6406d8f0 2024-04-25T19:05:47.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.003+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fee64070250 con 0x7fee6406d8f0 2024-04-25T19:05:47.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.004+0000 7fee60c00640 1 -- 172.21.15.123:0/1909030480 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7fee5805f910 con 0x7fee6406d8f0 2024-04-25T19:05:47.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:05:47.006 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","modified":"2024-04-25T19:01:25.058398Z","created":"2024-04-25T19:01:25.058398Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3301","nonce":0},{"type":"v1","addr":"172.21.15.146:6790","nonce":0}]},"addr":"172.21.15.146:6790/0","public_addr":"172.21.15.146:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T19:05:47.006 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-04-25T19:05:47.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 msgr2=0x7fee440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 0x7fee440769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee50000f60 tx=0x7fee50007a80 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 msgr2=0x7fee6414e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b3e0 tx=0x7fee5800b8a0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 shutdown_connections 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee64161b10 0x7fee6415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee640d1120 0x7fee6409b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 --2- 172.21.15.123:0/1909030480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee6406d8f0 0x7fee6414e010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.006+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 >> 172.21.15.123:0/1909030480 conn(0x7fee640d1ae0 msgr2=0x7fee64158160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.007+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 shutdown_connections 2024-04-25T19:05:47.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.007+0000 7fee6b400640 1 -- 172.21.15.123:0/1909030480 wait complete. 2024-04-25T19:05:47.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-04-25T19:05:47.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.276+0000 7ff996e00640 1 Processor -- start 2024-04-25T19:05:47.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.276+0000 7ff996e00640 1 -- start start 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.278+0000 7ff996e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9900b0ee0 0x7ff990160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.278+0000 7ff996e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900ce420 0x7ff9900ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.278+0000 7ff996e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff990160740 con 0x7ff9900ced70 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.278+0000 7ff996e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff9900d26a0 con 0x7ff9900b0ee0 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff996e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900d1a90 con 0x7ff9900ce420 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9900b0ee0 0x7ff990160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9900b0ee0 0x7ff990160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34722/0 (socket says 172.21.15.123:34722) 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 -- 172.21.15.123:0/2247314713 learned_addr learned my addr 172.21.15.123:0/2247314713 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995000640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900ce420 0x7ff9900ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 -- 172.21.15.123:0/2247314713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900ce420 msgr2=0x7ff9900ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900ce420 0x7ff9900ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 -- 172.21.15.123:0/2247314713 >> v1:172.21.15.146:6789/0 conn(0x7ff9900ced70 legacy=0x7ff990163e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.279+0000 7ff995a00640 1 -- 172.21.15.123:0/2247314713 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9900d1d70 con 0x7ff9900b0ee0 2024-04-25T19:05:47.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.280+0000 7ff995a00640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900b0ee0 0x7ff990160200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff984009870 tx=0x7ff98402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3557449393520726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.281+0000 7ff98fe00640 1 -- 172.21.15.123:0/2247314713 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984037070 con 0x7ff9900b0ee0 2024-04-25T19:05:47.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.281+0000 7ff98fe00640 1 -- 172.21.15.123:0/2247314713 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff984032440 con 0x7ff9900b0ee0 2024-04-25T19:05:47.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.281+0000 7ff98fe00640 1 -- 172.21.15.123:0/2247314713 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984032810 con 0x7ff9900b0ee0 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 -- 172.21.15.123:0/2247314713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900b0ee0 msgr2=0x7ff990160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900b0ee0 0x7ff990160200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff984009870 tx=0x7ff98402a210 comp rx=0 tx=0).stop 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 -- 172.21.15.123:0/2247314713 shutdown_connections 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900ce420 0x7ff9900ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 --2- 172.21.15.123:0/2247314713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900b0ee0 0x7ff990160200 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 -- 172.21.15.123:0/2247314713 >> 172.21.15.123:0/2247314713 conn(0x7ff99009b4a0 msgr2=0x7ff9900b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 -- 172.21.15.123:0/2247314713 shutdown_connections 2024-04-25T19:05:47.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.282+0000 7ff996e00640 1 -- 172.21.15.123:0/2247314713 wait complete. 2024-04-25T19:05:47.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.285+0000 7ff996e00640 1 Processor -- start 2024-04-25T19:05:47.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.285+0000 7ff996e00640 1 -- start start 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900b0ee0 0x7ff9900c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9900ced70 0x7ff99015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900b12f0 con 0x7ff9900ce420 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900d2330 con 0x7ff9900ced70 2024-04-25T19:05:47.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff996e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900cf180 con 0x7ff9900b0ee0 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff995000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff995000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34736/0 (socket says 172.21.15.123:34736) 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.287+0000 7ff995000640 1 -- 172.21.15.123:0/1960780485 learned_addr learned my addr 172.21.15.123:0/1960780485 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995a00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900b0ee0 0x7ff9900c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff996400640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9900ced70 0x7ff99015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 -- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900b0ee0 msgr2=0x7ff9900c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900b0ee0 0x7ff9900c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 -- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9900ced70 msgr2=0x7ff99015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9900ced70 0x7ff99015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 -- 172.21.15.123:0/1960780485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9900ca550 con 0x7ff9900ce420 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.288+0000 7ff995000640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c7f0 tx=0x7ff97c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.289+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c015070 con 0x7ff9900ce420 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.289+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97c004220 con 0x7ff9900ce420 2024-04-25T19:05:47.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.289+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c007620 con 0x7ff9900ce420 2024-04-25T19:05:47.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.289+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9900ca7d0 con 0x7ff9900ce420 2024-04-25T19:05:47.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.289+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9900cad90 con 0x7ff9900ce420 2024-04-25T19:05:47.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.290+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990050740 con 0x7ff9900ce420 2024-04-25T19:05:47.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.292+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff97c0079d0 con 0x7ff9900ce420 2024-04-25T19:05:47.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.293+0000 7ff98d600640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 0x7ff9700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.293+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7ff97c095410 con 0x7ff9900ce420 2024-04-25T19:05:47.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.294+0000 7ff995a00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 0x7ff9700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.294+0000 7ff995a00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 0x7ff9700769f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff984034000 tx=0x7ff98402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.298+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9900ca550 con 0x7ff9900ce420 2024-04-25T19:05:47.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.649+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) -- 0x7ff9900c8690 con 0x7ff9900ce420 2024-04-25T19:05:47.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.649+0000 7ff98d600640 1 -- 172.21.15.123:0/1960780485 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v11) ==== 92+0+63 (secure 0 0 0) 0x7ff97c05f990 con 0x7ff9900ce420 2024-04-25T19:05:47.651 INFO:teuthology.orchestra.run.smithi123.stdout:[client.admin] 2024-04-25T19:05:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCFqCpmzbhAHRAA26anAytsUh/6ELHsMn43Pg== 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.651+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 msgr2=0x7ff9700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.651+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 0x7ff9700769f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff984034000 tx=0x7ff98402e000 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.651+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 msgr2=0x7ff9900c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.651+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c7f0 tx=0x7ff97c00ccb0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 shutdown_connections 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff9700744e0 0x7ff9700769f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9900ced70 0x7ff99015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9900ce420 0x7ff9900c8140 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 --2- 172.21.15.123:0/1960780485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9900b0ee0 0x7ff9900c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 >> 172.21.15.123:0/1960780485 conn(0x7ff99009b4a0 msgr2=0x7ff990071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 shutdown_connections 2024-04-25T19:05:47.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:05:47.652+0000 7ff996e00640 1 -- 172.21.15.123:0/1960780485 wait complete. 2024-04-25T19:05:47.679 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:05:47.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/tmp/tmp.s4sluTBO7Yxfstests-dev/local.config 2024-04-25T19:05:47.743 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.123:6789:/test export TEST_DIR=/tmp/tmp.iUM1CQACHytest export SCRATCH_DEV=172.21.15.123:6789:/scratch export SCRATCH_MNT=/tmp/tmp.L6JI355wCGscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQCFqCpmzbhAHRAA26anAytsUh/6ELHsMn43Pg==" 2024-04-25T19:05:47.743 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:05:47.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/tmp/tmp.s4sluTBO7Yxfstests-dev/ceph.exclude 2024-04-25T19:05:47.846 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.s4sluTBO7Yxfstests-dev && exec sudo make) 2024-04-25T19:05:47.961 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize -cfi 2024-04-25T19:05:48.567 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T19:05:48.567 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './config.guess' 2024-04-25T19:05:48.582 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './config.sub' 2024-04-25T19:05:48.599 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './install-sh' 2024-04-25T19:05:48.614 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T19:05:48.705 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T19:05:48.705 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T19:05:48.724 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T19:05:48.746 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T19:05:48.769 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T19:05:48.791 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T19:05:48.819 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T19:05:48.823 INFO:teuthology.orchestra.run.smithi123.stdout:cp include/install-sh . 2024-04-25T19:05:48.827 INFO:teuthology.orchestra.run.smithi123.stdout:aclocal -I m4 2024-04-25T19:05:50.424 INFO:teuthology.orchestra.run.smithi123.stdout:autoheader 2024-04-25T19:05:50.826 INFO:teuthology.orchestra.run.smithi123.stdout:autoconf 2024-04-25T19:05:51.090 INFO:teuthology.orchestra.run.smithi123.stdout:./configure \ 2024-04-25T19:05:51.090 INFO:teuthology.orchestra.run.smithi123.stdout: --libexecdir=/usr/lib \ 2024-04-25T19:05:51.090 INFO:teuthology.orchestra.run.smithi123.stdout: --exec_prefix=/var/lib 2024-04-25T19:05:51.731 INFO:teuthology.orchestra.run.smithi123.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T19:05:51.732 INFO:teuthology.orchestra.run.smithi123.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T19:05:51.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to print strings... printf 2024-04-25T19:05:51.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc... gcc 2024-04-25T19:05:51.950 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether the C compiler works... yes 2024-04-25T19:05:51.950 INFO:teuthology.orchestra.run.smithi123.stdout:checking for C compiler default output file name... a.out 2024-04-25T19:05:52.025 INFO:teuthology.orchestra.run.smithi123.stdout:checking for suffix of executables... 2024-04-25T19:05:52.165 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are cross compiling... no 2024-04-25T19:05:52.215 INFO:teuthology.orchestra.run.smithi123.stdout:checking for suffix of object files... o 2024-04-25T19:05:52.258 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T19:05:52.304 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc accepts -g... yes 2024-04-25T19:05:52.403 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T19:05:52.413 INFO:teuthology.orchestra.run.smithi123.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-25T19:05:52.420 INFO:teuthology.orchestra.run.smithi123.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-25T19:05:52.425 INFO:teuthology.orchestra.run.smithi123.stdout:checking for egrep... /bin/grep -E 2024-04-25T19:05:52.431 INFO:teuthology.orchestra.run.smithi123.stdout:checking for fgrep... /bin/grep -F 2024-04-25T19:05:52.444 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ld used by gcc... /bin/ld 2024-04-25T19:05:52.450 INFO:teuthology.orchestra.run.smithi123.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-25T19:05:52.462 INFO:teuthology.orchestra.run.smithi123.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-25T19:05:53.521 INFO:teuthology.orchestra.run.smithi123.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-25T19:05:53.521 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether ln -s works... yes 2024-04-25T19:05:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T19:05:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T19:05:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T19:05:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:checking for /bin/ld option to reload object files... -r 2024-04-25T19:05:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:checking for objdump... objdump 2024-04-25T19:05:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T19:05:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dlltool... no 2024-04-25T19:05:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T19:05:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ar... ar 2024-04-25T19:05:53.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:53.627 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:53.677 INFO:teuthology.orchestra.run.smithi123.stdout:checking for archiver @FILE support... @ 2024-04-25T19:05:53.677 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strip... strip 2024-04-25T19:05:53.678 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ranlib... ranlib 2024-04-25T19:05:53.679 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gawk... gawk 2024-04-25T19:05:53.833 INFO:teuthology.orchestra.run.smithi123.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-25T19:05:53.837 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sysroot... no 2024-04-25T19:05:53.847 INFO:teuthology.orchestra.run.smithi123.stdout:checking for a working dd... /bin/dd 2024-04-25T19:05:53.856 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-25T19:05:53.894 INFO:teuthology.orchestra.run.smithi123.stdout:checking for mt... no 2024-04-25T19:05:53.901 INFO:teuthology.orchestra.run.smithi123.stdout:checking if : is a manifest tool... no 2024-04-25T19:05:54.008 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T19:05:54.337 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ANSI C header files... yes 2024-04-25T19:05:54.398 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/types.h... yes 2024-04-25T19:05:54.462 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/stat.h... yes 2024-04-25T19:05:54.527 INFO:teuthology.orchestra.run.smithi123.stdout:checking for stdlib.h... yes 2024-04-25T19:05:54.585 INFO:teuthology.orchestra.run.smithi123.stdout:checking for string.h... yes 2024-04-25T19:05:54.647 INFO:teuthology.orchestra.run.smithi123.stdout:checking for memory.h... yes 2024-04-25T19:05:54.716 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strings.h... yes 2024-04-25T19:05:54.778 INFO:teuthology.orchestra.run.smithi123.stdout:checking for inttypes.h... yes 2024-04-25T19:05:54.842 INFO:teuthology.orchestra.run.smithi123.stdout:checking for stdint.h... yes 2024-04-25T19:05:54.906 INFO:teuthology.orchestra.run.smithi123.stdout:checking for unistd.h... yes 2024-04-25T19:05:54.969 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dlfcn.h... yes 2024-04-25T19:05:54.980 INFO:teuthology.orchestra.run.smithi123.stdout:checking for objdir... .libs 2024-04-25T19:05:55.137 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T19:05:55.137 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T19:05:55.182 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T19:05:55.248 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc static flag -static works... no 2024-04-25T19:05:55.316 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T19:05:55.316 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T19:05:55.336 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T19:05:55.396 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T19:05:55.520 INFO:teuthology.orchestra.run.smithi123.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T19:05:55.520 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T19:05:55.523 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether stripping libraries is possible... yes 2024-04-25T19:05:55.523 INFO:teuthology.orchestra.run.smithi123.stdout:checking if libtool supports shared libraries... yes 2024-04-25T19:05:55.523 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether to build shared libraries... yes 2024-04-25T19:05:55.523 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether to build static libraries... yes 2024-04-25T19:05:55.538 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc... (cached) gcc 2024-04-25T19:05:55.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T19:05:55.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T19:05:55.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T19:05:55.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T19:05:55.582 INFO:teuthology.orchestra.run.smithi123.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T19:05:55.583 INFO:teuthology.orchestra.run.smithi123.stdout:checking for tar... /bin/tar 2024-04-25T19:05:55.583 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gzip... /bin/gzip 2024-04-25T19:05:55.615 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc -MM is supported... yes 2024-04-25T19:05:55.616 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sort... /bin/sort 2024-04-25T19:05:55.616 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether ln -s works... yes 2024-04-25T19:05:55.617 INFO:teuthology.orchestra.run.smithi123.stdout:checking for rpm... /bin/rpm 2024-04-25T19:05:55.627 INFO:teuthology.orchestra.run.smithi123.stdout:checking for rpmbuild... no 2024-04-25T19:05:55.683 INFO:teuthology.orchestra.run.smithi123.stdout:checking assert.h usability... yes 2024-04-25T19:05:55.708 INFO:teuthology.orchestra.run.smithi123.stdout:checking assert.h presence... yes 2024-04-25T19:05:55.708 INFO:teuthology.orchestra.run.smithi123.stdout:checking for assert.h... yes 2024-04-25T19:05:55.785 INFO:teuthology.orchestra.run.smithi123.stdout:checking bstring.h usability... no 2024-04-25T19:05:55.827 INFO:teuthology.orchestra.run.smithi123.stdout:checking bstring.h presence... no 2024-04-25T19:05:55.828 INFO:teuthology.orchestra.run.smithi123.stdout:checking for bstring.h... no 2024-04-25T19:05:55.893 INFO:teuthology.orchestra.run.smithi123.stdout:checking libgen.h usability... yes 2024-04-25T19:05:55.915 INFO:teuthology.orchestra.run.smithi123.stdout:checking libgen.h presence... yes 2024-04-25T19:05:55.915 INFO:teuthology.orchestra.run.smithi123.stdout:checking for libgen.h... yes 2024-04-25T19:05:55.986 INFO:teuthology.orchestra.run.smithi123.stdout:checking dirent.h usability... yes 2024-04-25T19:05:56.015 INFO:teuthology.orchestra.run.smithi123.stdout:checking dirent.h presence... yes 2024-04-25T19:05:56.015 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dirent.h... yes 2024-04-25T19:05:56.121 INFO:teuthology.orchestra.run.smithi123.stdout:checking errno.h usability... yes 2024-04-25T19:05:56.148 INFO:teuthology.orchestra.run.smithi123.stdout:checking errno.h presence... yes 2024-04-25T19:05:56.148 INFO:teuthology.orchestra.run.smithi123.stdout:checking for errno.h... yes 2024-04-25T19:05:56.225 INFO:teuthology.orchestra.run.smithi123.stdout:checking malloc.h usability... yes 2024-04-25T19:05:56.254 INFO:teuthology.orchestra.run.smithi123.stdout:checking malloc.h presence... yes 2024-04-25T19:05:56.254 INFO:teuthology.orchestra.run.smithi123.stdout:checking for malloc.h... yes 2024-04-25T19:05:56.333 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid.h usability... no 2024-04-25T19:05:56.372 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid.h presence... no 2024-04-25T19:05:56.372 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid.h... no 2024-04-25T19:05:56.441 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid/uuid.h usability... yes 2024-04-25T19:05:56.471 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid/uuid.h presence... yes 2024-04-25T19:05:56.471 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid/uuid.h... yes 2024-04-25T19:05:56.551 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/uuid.h usability... no 2024-04-25T19:05:56.593 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/uuid.h presence... no 2024-04-25T19:05:56.593 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/uuid.h... no 2024-04-25T19:05:56.667 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/file.h usability... yes 2024-04-25T19:05:56.697 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/file.h presence... yes 2024-04-25T19:05:56.698 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/file.h... yes 2024-04-25T19:05:56.773 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fcntl.h usability... yes 2024-04-25T19:05:56.803 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fcntl.h presence... yes 2024-04-25T19:05:56.803 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fcntl.h... yes 2024-04-25T19:05:56.886 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/syssgi.h usability... no 2024-04-25T19:05:56.924 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/syssgi.h presence... no 2024-04-25T19:05:56.924 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/syssgi.h... no 2024-04-25T19:05:57.013 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/param.h usability... yes 2024-04-25T19:05:57.047 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/param.h presence... yes 2024-04-25T19:05:57.047 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/param.h... yes 2024-04-25T19:05:57.065 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T19:05:57.140 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/statvfs.h usability... yes 2024-04-25T19:05:57.165 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/statvfs.h presence... yes 2024-04-25T19:05:57.165 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/statvfs.h... yes 2024-04-25T19:05:57.246 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/time.h usability... yes 2024-04-25T19:05:57.275 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/time.h presence... yes 2024-04-25T19:05:57.275 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/time.h... yes 2024-04-25T19:05:57.366 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/ioctl.h usability... yes 2024-04-25T19:05:57.393 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/ioctl.h presence... yes 2024-04-25T19:05:57.393 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/ioctl.h... yes 2024-04-25T19:05:57.477 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/wait.h usability... yes 2024-04-25T19:05:57.508 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/wait.h presence... yes 2024-04-25T19:05:57.508 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/wait.h... yes 2024-04-25T19:05:57.526 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/types.h... (cached) yes 2024-04-25T19:05:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strings.h... (cached) yes 2024-04-25T19:05:57.613 INFO:teuthology.orchestra.run.smithi123.stdout:checking err.h usability... yes 2024-04-25T19:05:57.640 INFO:teuthology.orchestra.run.smithi123.stdout:checking err.h presence... yes 2024-04-25T19:05:57.641 INFO:teuthology.orchestra.run.smithi123.stdout:checking for err.h... yes 2024-04-25T19:05:57.717 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/falloc.h usability... yes 2024-04-25T19:05:57.742 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/falloc.h presence... yes 2024-04-25T19:05:57.743 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/falloc.h... yes 2024-04-25T19:05:57.819 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T19:05:57.862 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T19:05:57.862 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T19:05:57.933 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T19:05:57.972 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T19:05:57.972 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T19:05:58.042 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T19:05:58.082 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T19:05:58.082 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/platform_defs.h... no 2024-04-25T19:05:58.153 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T19:05:58.191 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T19:05:58.191 INFO:teuthology.orchestra.run.smithi123.stdout:checking for btrfs/ioctl.h... no 2024-04-25T19:05:58.260 INFO:teuthology.orchestra.run.smithi123.stdout:checking cifs/ioctl.h usability... no 2024-04-25T19:05:58.301 INFO:teuthology.orchestra.run.smithi123.stdout:checking cifs/ioctl.h presence... no 2024-04-25T19:05:58.302 INFO:teuthology.orchestra.run.smithi123.stdout:checking for cifs/ioctl.h... no 2024-04-25T19:05:58.379 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/mman.h usability... yes 2024-04-25T19:05:58.409 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/mman.h presence... yes 2024-04-25T19:05:58.409 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/mman.h... yes 2024-04-25T19:05:58.492 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/ext4.h usability... no 2024-04-25T19:05:58.533 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/ext4.h presence... no 2024-04-25T19:05:58.533 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/ext4.h... no 2024-04-25T19:05:58.580 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T19:05:58.684 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid_compare... no 2024-04-25T19:05:58.760 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T19:05:58.889 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xfs.h... yes 2024-04-25T19:05:58.940 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/libxfs.h... no 2024-04-25T19:05:59.000 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T19:05:59.079 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xqm.h... yes 2024-04-25T19:05:59.181 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T19:05:59.241 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/handle.h usability... yes 2024-04-25T19:05:59.266 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/handle.h presence... yes 2024-04-25T19:05:59.266 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/handle.h... yes 2024-04-25T19:05:59.353 INFO:teuthology.orchestra.run.smithi123.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T19:05:59.441 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/acl.h usability... yes 2024-04-25T19:05:59.470 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/acl.h presence... yes 2024-04-25T19:05:59.471 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/acl.h... yes 2024-04-25T19:05:59.549 INFO:teuthology.orchestra.run.smithi123.stdout:checking acl/libacl.h usability... yes 2024-04-25T19:05:59.581 INFO:teuthology.orchestra.run.smithi123.stdout:checking acl/libacl.h presence... yes 2024-04-25T19:05:59.581 INFO:teuthology.orchestra.run.smithi123.stdout:checking for acl/libacl.h... yes 2024-04-25T19:05:59.665 INFO:teuthology.orchestra.run.smithi123.stdout:checking for acl_init in -lacl... yes 2024-04-25T19:05:59.730 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T19:05:59.773 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T19:05:59.773 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gdbm-ndbm.h... no 2024-04-25T19:05:59.837 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm/ndbm.h usability... yes 2024-04-25T19:05:59.867 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm/ndbm.h presence... yes 2024-04-25T19:05:59.867 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gdbm/ndbm.h... yes 2024-04-25T19:05:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_open in -lgdbm... no 2024-04-25T19:06:00.053 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-25T19:06:00.138 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_store in -lgdbm... no 2024-04-25T19:06:00.229 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_close in -lgdbm... no 2024-04-25T19:06:00.321 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-25T19:06:00.402 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-25T19:06:00.486 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-25T19:06:00.567 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-25T19:06:00.626 INFO:teuthology.orchestra.run.smithi123.stdout:checking libaio.h usability... yes 2024-04-25T19:06:00.656 INFO:teuthology.orchestra.run.smithi123.stdout:checking libaio.h presence... yes 2024-04-25T19:06:00.656 INFO:teuthology.orchestra.run.smithi123.stdout:checking for libaio.h... yes 2024-04-25T19:06:00.715 INFO:teuthology.orchestra.run.smithi123.stdout:checking liburing.h usability... no 2024-04-25T19:06:00.755 INFO:teuthology.orchestra.run.smithi123.stdout:checking liburing.h presence... no 2024-04-25T19:06:00.755 INFO:teuthology.orchestra.run.smithi123.stdout:checking for liburing.h... no 2024-04-25T19:06:00.821 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/capability.h usability... yes 2024-04-25T19:06:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/capability.h presence... yes 2024-04-25T19:06:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/capability.h... yes 2024-04-25T19:06:00.918 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fiemap.h usability... yes 2024-04-25T19:06:00.943 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fiemap.h presence... yes 2024-04-25T19:06:00.943 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/fiemap.h... yes 2024-04-25T19:06:01.023 INFO:teuthology.orchestra.run.smithi123.stdout:checking for fallocate... yes 2024-04-25T19:06:01.106 INFO:teuthology.orchestra.run.smithi123.stdout:checking for open_by_handle_at... yes 2024-04-25T19:06:01.167 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fs.h usability... yes 2024-04-25T19:06:01.197 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fs.h presence... yes 2024-04-25T19:06:01.197 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/fs.h... yes 2024-04-25T19:06:01.262 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfsutil.h usability... no 2024-04-25T19:06:01.304 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfsutil.h presence... no 2024-04-25T19:06:01.304 INFO:teuthology.orchestra.run.smithi123.stdout:checking for btrfsutil.h... no 2024-04-25T19:06:01.403 INFO:teuthology.orchestra.run.smithi123.stdout:checking for copy_file_range... yes 2024-04-25T19:06:01.493 INFO:teuthology.orchestra.run.smithi123.stdout:checking for SEEK_DATA... yes 2024-04-25T19:06:01.604 INFO:teuthology.orchestra.run.smithi123.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T19:06:01.695 INFO:teuthology.orchestra.run.smithi123.stdout:checking for nftw... yes 2024-04-25T19:06:01.787 INFO:teuthology.orchestra.run.smithi123.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T19:06:01.867 INFO:teuthology.orchestra.run.smithi123.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T19:06:01.953 INFO:teuthology.orchestra.run.smithi123.stdout:checking for FICLONE... yes 2024-04-25T19:06:02.041 INFO:teuthology.orchestra.run.smithi123.stdout:checking for renameat2... yes 2024-04-25T19:06:02.124 INFO:teuthology.orchestra.run.smithi123.stdout:checking for reallocarray... yes 2024-04-25T19:06:02.216 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct mount_attr... yes 2024-04-25T19:06:02.322 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T19:06:02.424 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T19:06:02.533 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T19:06:02.640 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T19:06:02.749 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T19:06:02.855 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T19:06:02.962 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T19:06:03.036 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs.h usability... yes 2024-04-25T19:06:03.065 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs.h presence... yes 2024-04-25T19:06:03.066 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/btrfs.h... yes 2024-04-25T19:06:03.143 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T19:06:03.172 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T19:06:03.173 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T19:06:03.239 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T19:06:03.308 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T19:06:03.349 INFO:teuthology.orchestra.run.smithi123.stdout:configure: creating ./config.status 2024-04-25T19:06:04.523 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: creating include/builddefs 2024-04-25T19:06:04.564 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: creating include/config.h 2024-04-25T19:06:04.595 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: executing libtool commands 2024-04-25T19:06:04.711 INFO:teuthology.orchestra.run.smithi123.stdout:Building include 2024-04-25T19:06:04.721 INFO:teuthology.orchestra.run.smithi123.stdout:Building lib 2024-04-25T19:06:04.730 INFO:teuthology.orchestra.run.smithi123.stdout: [LTDEP] 2024-04-25T19:06:05.057 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dataascii.lo 2024-04-25T19:06:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] databin.lo 2024-04-25T19:06:05.619 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] datapid.lo 2024-04-25T19:06:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] file_lock.lo 2024-04-25T19:06:06.153 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] forker.lo 2024-04-25T19:06:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] pattern.lo 2024-04-25T19:06:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] open_flags.lo 2024-04-25T19:06:06.949 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] random_range.lo 2024-04-25T19:06:07.312 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] string_to_tokens.lo 2024-04-25T19:06:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] str_to_bytes.lo 2024-04-25T19:06:07.843 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] tlibio.lo 2024-04-25T19:06:08.249 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] write_log.lo 2024-04-25T19:06:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] random.lo 2024-04-25T19:06:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: [LD] 2024-04-25T19:06:09.073 INFO:teuthology.orchestra.run.smithi123.stdout:Building ltp 2024-04-25T19:06:09.082 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:09.370 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] doio 2024-04-25T19:06:10.255 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsstress 2024-04-25T19:06:11.865 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsx 2024-04-25T19:06:12.777 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] growfiles 2024-04-25T19:06:13.487 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] iogen 2024-04-25T19:06:14.036 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-stress 2024-04-25T19:06:14.657 INFO:teuthology.orchestra.run.smithi123.stdout:Building src 2024-04-25T19:06:14.671 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:18.638 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:18.716 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:18.740 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:19.224 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-25T19:06:19.281 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirstress 2024-04-25T19:06:19.698 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fill 2024-04-25T19:06:19.993 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fill2 2024-04-25T19:06:20.374 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] getpagesize 2024-04-25T19:06:20.675 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] holes 2024-04-25T19:06:21.033 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] lstat64 2024-04-25T19:06:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] nametest 2024-04-25T19:06:21.799 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] permname 2024-04-25T19:06:22.192 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] randholes 2024-04-25T19:06:22.625 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] runas 2024-04-25T19:06:22.970 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] truncfile 2024-04-25T19:06:23.327 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] usemem 2024-04-25T19:06:23.640 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmapcat 2024-04-25T19:06:23.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:23.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:23.929 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] append_reader 2024-04-25T19:06:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] append_writer 2024-04-25T19:06:24.570 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirperf 2024-04-25T19:06:24.995 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] metaperf 2024-04-25T19:06:25.480 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] devzero 2024-04-25T19:06:25.842 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] feature 2024-04-25T19:06:26.226 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] alloc 2024-04-25T19:06:26.620 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fault 2024-04-25T19:06:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fstest 2024-04-25T19:06:27.388 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_access_root 2024-04-25T19:06:27.691 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] godown 2024-04-25T19:06:28.017 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] resvtest 2024-04-25T19:06:28.370 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] writemod 2024-04-25T19:06:28.675 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] writev_on_pagefault 2024-04-25T19:06:29.002 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] makeextents 2024-04-25T19:06:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] itrash 2024-04-25T19:06:29.642 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] rename 2024-04-25T19:06:29.944 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] multi_open_unlink 2024-04-25T19:06:30.271 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] unwritten_sync 2024-04-25T19:06:30.628 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] genhashnames 2024-04-25T19:06:31.003 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_holes 2024-04-25T19:06:31.330 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_writev 2024-04-25T19:06:31.649 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_truncate_cmtime 2024-04-25T19:06:31.990 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirhash_collide 2024-04-25T19:06:32.360 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_rename_overwrite 2024-04-25T19:06:32.659 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] holetest 2024-04-25T19:06:33.039 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_truncate_self 2024-04-25T19:06:33.347 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] af_unix 2024-04-25T19:06:33.683 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_stale_pmd 2024-04-25T19:06:33.998 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_cow_race 2024-04-25T19:06:34.339 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_fallocate 2024-04-25T19:06:34.665 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsync-err 2024-04-25T19:06:35.030 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_write_ro 2024-04-25T19:06:35.355 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T19:06:35.664 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T19:06:35.984 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ofd_locks 2024-04-25T19:06:36.357 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_collision 2024-04-25T19:06:36.723 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmap-write-concurrent 2024-04-25T19:06:37.075 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_get_file_time 2024-04-25T19:06:37.388 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_create_short_dirs 2024-04-25T19:06:37.700 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_create_long_dirs 2024-04-25T19:06:38.044 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_enospc 2024-04-25T19:06:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_writev_overlap 2024-04-25T19:06:38.748 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] checkpoint_journal 2024-04-25T19:06:39.093 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmap-rw-fault 2024-04-25T19:06:39.447 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] allocstale 2024-04-25T19:06:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T19:06:40.128 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fake-dump-rootino 2024-04-25T19:06:40.452 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-buf-fault 2024-04-25T19:06:40.782 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] rewinddir-test 2024-04-25T19:06:41.113 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] readdir-while-renames 2024-04-25T19:06:41.424 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] xfsctl 2024-04-25T19:06:41.788 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bstat 2024-04-25T19:06:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mtab 2024-04-25T19:06:42.559 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] getdevicesize 2024-04-25T19:06:42.867 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T19:06:43.211 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T19:06:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] ftrunc 2024-04-25T19:06:43.872 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] trunc 2024-04-25T19:06:44.207 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fs_perms 2024-04-25T19:06:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] testx 2024-04-25T19:06:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] looptest 2024-04-25T19:06:45.177 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] locktest 2024-04-25T19:06:45.669 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] unwritten_mmap 2024-04-25T19:06:46.009 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_unlink_test 2024-04-25T19:06:46.389 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] deduperace 2024-04-25T19:06:46.761 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T19:06:47.147 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_offset 2024-04-25T19:06:47.463 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_futimens 2024-04-25T19:06:47.771 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_immutable 2024-04-25T19:06:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] stale_handle 2024-04-25T19:06:48.913 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] pwrite_mmap_blocked 2024-04-25T19:06:49.226 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_offset2 2024-04-25T19:06:49.587 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] seek_sanity_test 2024-04-25T19:06:50.232 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] seek_copy_test 2024-04-25T19:06:50.580 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_1 2024-04-25T19:06:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_2 2024-04-25T19:06:51.203 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsync-tester 2024-04-25T19:06:51.638 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] nsexec 2024-04-25T19:06:51.989 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] cloner 2024-04-25T19:06:52.335 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] renameat2 2024-04-25T19:06:52.678 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_getcwd 2024-04-25T19:06:53.014 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] e4compact 2024-04-25T19:06:53.374 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] test-nextquota 2024-04-25T19:06:53.724 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] punch-alternating 2024-04-25T19:06:53.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:53.886 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T19:06:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] listxattr 2024-04-25T19:06:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-interleaved 2024-04-25T19:06:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_type 2024-04-25T19:06:55.347 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-invalidate-cache 2024-04-25T19:06:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] stat_test 2024-04-25T19:06:56.201 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T19:06:56.535 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] attr_replace_test 2024-04-25T19:06:56.865 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] swapon 2024-04-25T19:06:57.187 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mkswap 2024-04-25T19:06:57.507 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_attr_corruption 2024-04-25T19:06:57.808 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_open_tmpfiles 2024-04-25T19:06:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fscrypt-crypt-util 2024-04-25T19:06:59.061 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_null_ocount 2024-04-25T19:06:59.366 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] splice-test 2024-04-25T19:06:59.708 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] chprojid_fail 2024-04-25T19:07:00.049 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] detached_mounts_propagation 2024-04-25T19:07:00.409 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] ext4_resize 2024-04-25T19:07:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_3 2024-04-25T19:07:01.047 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] splice2pipe 2024-04-25T19:07:01.382 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] uuid_ioctl 2024-04-25T19:07:01.695 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T19:07:02.039 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fiemap-fault 2024-04-25T19:07:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fiemap-tester 2024-04-25T19:07:02.830 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_stripealign 2024-04-25T19:07:03.164 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] open_by_handle 2024-04-25T19:07:03.597 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dbtest 2024-04-25T19:07:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_dio 2024-04-25T19:07:04.319 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] xfsfind 2024-04-25T19:07:04.703 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_reflink_read_race 2024-04-25T19:07:05.082 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fssum 2024-04-25T19:07:05.710 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-25T19:07:05.724 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] replay-log 2024-04-25T19:07:06.239 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-25T19:07:06.250 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-25T19:07:06.265 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aiocp 2024-04-25T19:07:06.669 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T19:07:07.010 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T19:07:07.344 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-cow-race 2024-04-25T19:07:07.664 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-cycle-write 2024-04-25T19:07:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-eof-race 2024-04-25T19:07:08.363 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-extend-stat 2024-04-25T19:07:08.693 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-fcntl-race 2024-04-25T19:07:09.039 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T19:07:09.356 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T19:07:09.697 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T19:07:10.035 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aiodio_sparse2 2024-04-25T19:07:10.423 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T19:07:10.736 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-write-verify 2024-04-25T19:07:11.106 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T19:07:11.392 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T19:07:11.689 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T19:07:12.035 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-25T19:07:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] vfstest 2024-04-25T19:07:15.984 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mount-idmapped 2024-04-25T19:07:16.561 INFO:teuthology.orchestra.run.smithi123.stdout:Building m4 2024-04-25T19:07:16.572 INFO:teuthology.orchestra.run.smithi123.stdout:Building common 2024-04-25T19:07:16.582 INFO:teuthology.orchestra.run.smithi123.stdout:Building tests 2024-04-25T19:07:16.595 INFO:teuthology.orchestra.run.smithi123.stdout:Building btrfs 2024-04-25T19:07:16.610 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/btrfs/group.list 2024-04-25T19:07:16.833 INFO:teuthology.orchestra.run.smithi123.stdout:Building ceph 2024-04-25T19:07:16.847 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ceph/group.list 2024-04-25T19:07:16.898 INFO:teuthology.orchestra.run.smithi123.stdout:Building cifs 2024-04-25T19:07:16.913 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/cifs/group.list 2024-04-25T19:07:16.957 INFO:teuthology.orchestra.run.smithi123.stdout:Building ext4 2024-04-25T19:07:16.972 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ext4/group.list 2024-04-25T19:07:17.116 INFO:teuthology.orchestra.run.smithi123.stdout:Building f2fs 2024-04-25T19:07:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/f2fs/group.list 2024-04-25T19:07:17.194 INFO:teuthology.orchestra.run.smithi123.stdout:Building generic 2024-04-25T19:07:17.212 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/generic/group.list 2024-04-25T19:07:17.576 INFO:teuthology.orchestra.run.smithi123.stdout:Building nfs 2024-04-25T19:07:17.591 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/nfs/group.list 2024-04-25T19:07:17.643 INFO:teuthology.orchestra.run.smithi123.stdout:Building ocfs2 2024-04-25T19:07:17.657 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ocfs2/group.list 2024-04-25T19:07:17.705 INFO:teuthology.orchestra.run.smithi123.stdout:Building overlay 2024-04-25T19:07:17.717 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/overlay/group.list 2024-04-25T19:07:17.864 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-25T19:07:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/perf/group.list 2024-04-25T19:07:17.919 INFO:teuthology.orchestra.run.smithi123.stdout:Building selftest 2024-04-25T19:07:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/selftest/group.list 2024-04-25T19:07:17.978 INFO:teuthology.orchestra.run.smithi123.stdout:Building shared 2024-04-25T19:07:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/shared/group.list 2024-04-25T19:07:18.055 INFO:teuthology.orchestra.run.smithi123.stdout:Building tmpfs 2024-04-25T19:07:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/tmpfs/group.list 2024-04-25T19:07:18.118 INFO:teuthology.orchestra.run.smithi123.stdout:Building udf 2024-04-25T19:07:18.132 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/udf/group.list 2024-04-25T19:07:18.174 INFO:teuthology.orchestra.run.smithi123.stdout:Building xfs 2024-04-25T19:07:18.192 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/xfs/group.list 2024-04-25T19:07:18.534 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.s4sluTBO7Yxfstests-dev && exec sudo make install) 2024-04-25T19:07:18.651 INFO:teuthology.orchestra.run.smithi123.stdout:Building include 2024-04-25T19:07:18.663 INFO:teuthology.orchestra.run.smithi123.stdout:Building lib 2024-04-25T19:07:18.673 INFO:teuthology.orchestra.run.smithi123.stdout:Building ltp 2024-04-25T19:07:18.685 INFO:teuthology.orchestra.run.smithi123.stdout:Building src 2024-04-25T19:07:18.769 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-25T19:07:18.778 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-25T19:07:18.788 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-25T19:07:18.799 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-25T19:07:18.810 INFO:teuthology.orchestra.run.smithi123.stdout:Building m4 2024-04-25T19:07:18.819 INFO:teuthology.orchestra.run.smithi123.stdout:Building common 2024-04-25T19:07:18.829 INFO:teuthology.orchestra.run.smithi123.stdout:Building tests 2024-04-25T19:07:18.843 INFO:teuthology.orchestra.run.smithi123.stdout:Building btrfs 2024-04-25T19:07:18.858 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/btrfs/group.list 2024-04-25T19:07:19.074 INFO:teuthology.orchestra.run.smithi123.stdout:Building ceph 2024-04-25T19:07:19.087 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ceph/group.list 2024-04-25T19:07:19.137 INFO:teuthology.orchestra.run.smithi123.stdout:Building cifs 2024-04-25T19:07:19.151 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/cifs/group.list 2024-04-25T19:07:19.196 INFO:teuthology.orchestra.run.smithi123.stdout:Building ext4 2024-04-25T19:07:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ext4/group.list 2024-04-25T19:07:19.344 INFO:teuthology.orchestra.run.smithi123.stdout:Building f2fs 2024-04-25T19:07:19.357 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/f2fs/group.list 2024-04-25T19:07:19.415 INFO:teuthology.orchestra.run.smithi123.stdout:Building generic 2024-04-25T19:07:19.432 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/generic/group.list 2024-04-25T19:07:19.792 INFO:teuthology.orchestra.run.smithi123.stdout:Building nfs 2024-04-25T19:07:19.808 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/nfs/group.list 2024-04-25T19:07:19.859 INFO:teuthology.orchestra.run.smithi123.stdout:Building ocfs2 2024-04-25T19:07:19.873 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ocfs2/group.list 2024-04-25T19:07:19.921 INFO:teuthology.orchestra.run.smithi123.stdout:Building overlay 2024-04-25T19:07:19.935 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/overlay/group.list 2024-04-25T19:07:20.086 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-25T19:07:20.099 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/perf/group.list 2024-04-25T19:07:20.137 INFO:teuthology.orchestra.run.smithi123.stdout:Building selftest 2024-04-25T19:07:20.150 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/selftest/group.list 2024-04-25T19:07:20.191 INFO:teuthology.orchestra.run.smithi123.stdout:Building shared 2024-04-25T19:07:20.205 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/shared/group.list 2024-04-25T19:07:20.268 INFO:teuthology.orchestra.run.smithi123.stdout:Building tmpfs 2024-04-25T19:07:20.282 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/tmpfs/group.list 2024-04-25T19:07:20.334 INFO:teuthology.orchestra.run.smithi123.stdout:Building udf 2024-04-25T19:07:20.348 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/udf/group.list 2024-04-25T19:07:20.389 INFO:teuthology.orchestra.run.smithi123.stdout:Building xfs 2024-04-25T19:07:20.407 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/xfs/group.list 2024-04-25T19:07:20.756 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-25T19:07:20.760 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T19:07:20.760 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-25T19:07:20.765 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T19:07:20.765 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-25T19:07:20.770 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-25T19:07:20.801 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-25T19:07:21.041 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-25T19:07:21.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-25T19:07:21.121 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-25T19:07:21.131 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-25T19:07:21.142 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-25T19:07:21.154 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-25T19:07:21.163 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C log-writes install 2024-04-25T19:07:21.166 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-25T19:07:21.196 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-25T19:07:21.225 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C perf install 2024-04-25T19:07:21.229 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-25T19:07:21.260 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-25T19:07:21.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-25T19:07:21.347 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-25T19:07:21.378 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-25T19:07:21.561 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C vfs install 2024-04-25T19:07:21.565 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-25T19:07:21.596 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-25T19:07:21.641 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-25T19:07:21.674 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-25T19:07:24.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:07:24.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:07:26.062 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-25T19:07:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-25T19:07:26.423 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-25T19:07:26.427 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T19:07:26.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-25T19:07:26.431 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-25T19:07:26.476 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-25T19:07:26.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-25T19:07:26.993 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/btrfs/ install 2024-04-25T19:07:27.000 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/btrfs/group.list 2024-04-25T19:07:27.219 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-25T19:07:27.254 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-25T19:07:30.680 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-25T19:07:30.714 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-25T19:07:34.054 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ceph/ install 2024-04-25T19:07:34.061 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ceph/group.list 2024-04-25T19:07:34.112 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-25T19:07:34.144 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-25T19:07:34.217 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-25T19:07:34.249 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-25T19:07:34.321 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/cifs/ install 2024-04-25T19:07:34.327 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/cifs/group.list 2024-04-25T19:07:34.369 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-25T19:07:34.400 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-25T19:07:34.432 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-25T19:07:34.464 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-25T19:07:34.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ext4/ install 2024-04-25T19:07:34.503 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ext4/group.list 2024-04-25T19:07:34.638 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-25T19:07:34.668 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-25T19:07:35.444 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-25T19:07:35.477 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-25T19:07:36.260 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/f2fs/ install 2024-04-25T19:07:36.267 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/f2fs/group.list 2024-04-25T19:07:36.327 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-25T19:07:36.359 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-25T19:07:36.401 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-25T19:07:36.434 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-25T19:07:36.478 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/generic/ install 2024-04-25T19:07:36.487 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/generic/group.list 2024-04-25T19:07:36.854 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-25T19:07:36.887 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-25T19:07:45.173 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-25T19:07:45.206 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-25T19:07:52.884 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/nfs/ install 2024-04-25T19:07:52.890 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/nfs/group.list 2024-04-25T19:07:52.941 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-25T19:07:52.973 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-25T19:07:53.005 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-25T19:07:53.037 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-25T19:07:53.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ocfs2/ install 2024-04-25T19:07:53.078 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/ocfs2/group.list 2024-04-25T19:07:53.124 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-25T19:07:53.154 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-25T19:07:53.185 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-25T19:07:53.218 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-25T19:07:53.251 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/overlay/ install 2024-04-25T19:07:53.259 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/overlay/group.list 2024-04-25T19:07:53.399 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-25T19:07:53.429 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-25T19:07:54.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:07:54.130 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:07:54.446 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-25T19:07:54.476 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-25T19:07:55.568 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/perf/ install 2024-04-25T19:07:55.574 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/perf/group.list 2024-04-25T19:07:55.614 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-25T19:07:55.646 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-25T19:07:55.678 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-25T19:07:55.709 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-25T19:07:55.742 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/selftest/ install 2024-04-25T19:07:55.749 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/selftest/group.list 2024-04-25T19:07:55.792 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-25T19:07:55.824 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-25T19:07:55.905 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-25T19:07:55.938 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-25T19:07:56.020 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/shared/ install 2024-04-25T19:07:56.027 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/shared/group.list 2024-04-25T19:07:56.089 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-25T19:07:56.120 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-25T19:07:56.171 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-25T19:07:56.202 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-25T19:07:56.254 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/tmpfs/ install 2024-04-25T19:07:56.261 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/tmpfs/group.list 2024-04-25T19:07:56.306 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-25T19:07:56.337 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-25T19:07:56.368 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-25T19:07:56.400 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-25T19:07:56.431 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/udf/ install 2024-04-25T19:07:56.438 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/udf/group.list 2024-04-25T19:07:56.478 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-25T19:07:56.510 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-25T19:07:56.542 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-25T19:07:56.574 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-25T19:07:56.608 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/xfs/ install 2024-04-25T19:07:56.616 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.s4sluTBO7Yxfstests-dev/tests/xfs/group.list 2024-04-25T19:07:56.962 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-25T19:07:56.993 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-25T19:08:04.455 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-25T19:08:04.487 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-25T19:08:12.286 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-25T19:08:12.322 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-25T19:08:12.355 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-25T19:08:12.387 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-25T19:08:12.428 INFO:tasks.cephfs.test_acls:client is kernel mounted 2024-04-25T19:08:12.428 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T19:08:12.429 DEBUG:teuthology.orchestra.run.smithi123:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.s4sluTBO7Yxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-25T19:08:13.131 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph 2024-04-25T19:08:13.146 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-25T19:08:13.148 INFO:teuthology.orchestra.run.smithi123.stdout:MKFS_OPTIONS -- 172.21.15.123:6789:/scratch 2024-04-25T19:08:13.152 INFO:teuthology.orchestra.run.smithi123.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQCFqCpmzbhAHRAA26anAytsUh/6ELHsMn43Pg== -o context=system_u:object_r:root_t:s0 172.21.15.123:6789:/scratch /tmp/tmp.L6JI355wCGscratch 2024-04-25T19:08:13.152 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:15.294 INFO:teuthology.orchestra.run.smithi123.stdout:generic/099 2s 2024-04-25T19:08:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/099 2024-04-25T19:08:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-25T19:08:15.318 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:15.428 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-04-25T19:08:15.428 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 MKFS_OPTIONS -- 172.21.15.123:6789:/scratch MOUNT_OPTIONS -- -o name=admin,secret=AQCFqCpmzbhAHRAA26anAytsUh/6ELHsMn43Pg== -o context=system_u:object_r:root_t:s0 172.21.15.123:6789:/scratch /tmp/tmp.L6JI355wCGscratch generic/099 2s Ran: generic/099 Passed all 1 tests 2024-04-25T19:08:15.428 INFO:tasks.cephfs.test_acls:Command stderr - 2024-04-25T19:08:15.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo userdel --force --remove fsgqa 2024-04-25T19:08:16.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo userdel --force --remove 123456-fsgqa 2024-04-25T19:08:16.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo groupdel fsgqa 2024-04-25T19:08:16.861 INFO:teuthology.orchestra.run.smithi123.stderr:groupdel: group 'fsgqa' does not exist 2024-04-25T19:08:16.865 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T19:08:16.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:16.866 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.s4sluTBO7Yxfstests-dev/results) 2024-04-25T19:08:16.947 INFO:teuthology.orchestra.run.smithi123.stdout: File: /tmp/tmp.s4sluTBO7Yxfstests-dev/results 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 4096 Blocks: 8 IO Block: 4096 directory 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 801h/2049d Inode: 555876 Links: 3 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Context: unconfined_u:object_r:user_tmp_t:s0 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-25 19:08:13.076092257 +0000 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-25 19:08:15.304040519 +0000 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-25 19:08:15.304040519 +0000 2024-04-25T19:08:16.948 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-25 19:08:13.076092257 +0000 2024-04-25T19:08:16.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:16.952 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p /var/log/ceph/xfstests-dev-results) 2024-04-25T19:08:17.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:17.037 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -r /tmp/tmp.s4sluTBO7Yxfstests-dev/results /var/log/ceph/xfstests-dev-results) 2024-04-25T19:08:17.122 INFO:tasks.cephfs.xfstests_dev:results dir from xfstests-dev has been saved; it was copied from /tmp/tmp.s4sluTBO7Yxfstests-dev to /var/log/ceph. 2024-04-25T19:08:17.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf /tmp/tmp.s4sluTBO7Yxfstests-dev 2024-04-25T19:08:17.434 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T19:08:17.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T19:08:17.534 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T19:08:17.659 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T19:08:17.659 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T19:08:17.659 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:08:17.659 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:17.715 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:08:17.715 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:17.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:08:18.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.049+0000 7f7967e00640 1 Processor -- start 2024-04-25T19:08:18.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.049+0000 7f7967e00640 1 -- start start 2024-04-25T19:08:18.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7967e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f79681450d0 0x7f796806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7967e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796806ded0 con 0x7f796806dac0 2024-04-25T19:08:18.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7967e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79681456e0 con 0x7f79680b0cc0 2024-04-25T19:08:18.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7967e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f79681462f0 con 0x7f79681450d0 2024-04-25T19:08:18.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f795e000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f796806dac0 0x7f79680b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42448/0 (socket says 172.21.15.123:42448) 2024-04-25T19:08:18.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f795e000640 1 -- 172.21.15.123:0/1410142608 learned_addr learned my addr 172.21.15.123:0/1410142608 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966a00640 1 --2- 172.21.15.123:0/1410142608 >> v2:172.21.15.146:3300/0 conn(0x7f79681450d0 0x7f796806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3115641678 0 0) 0x7f7950001110 con 0x7f796806dac0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f794c003520 con 0x7f796806dac0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918488052 0 0) 0x7f794c003520 con 0x7f796806dac0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 >> v2:172.21.15.146:3300/0 conn(0x7f79681450d0 msgr2=0x7f796806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 --2- 172.21.15.123:0/1410142608 >> v2:172.21.15.146:3300/0 conn(0x7f79681450d0 0x7f796806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 >> v1:172.21.15.146:6789/0 conn(0x7f79680b0cc0 legacy=0x7f7968146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79681459c0 con 0x7f796806dac0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f7950001840 con 0x7f796806dac0 2024-04-25T19:08:18.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 >> v1:172.21.15.123:6789/0 conn(0x7f796806dac0 legacy=0x7f79680b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 0x7f794c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f794c004800 0x7f794c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c005350 0x7f794c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79681462f0 con 0x7f794c0039b0 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f794c00c230 con 0x7f794c005350 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f794c00c430 con 0x7f794c004800 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.052+0000 7f7966a00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 0x7f794c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7966a00640 1 -- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f794c004800 msgr2=0x7f794c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7966a00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f794c004800 0x7f794c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7966a00640 1 -- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c005350 msgr2=0x7f794c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7966a00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c005350 0x7f794c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7966a00640 1 -- 172.21.15.123:0/1410142608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c003520 con 0x7f794c0039b0 2024-04-25T19:08:18.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.053+0000 7f7967400640 1 -- 172.21.15.123:0/1410142608 reap_dead start 2024-04-25T19:08:18.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.054+0000 7f7966a00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 0x7f794c003e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f795800ba30 tx=0x7f794c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26befa896d69734a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.054+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958030070 con 0x7f794c0039b0 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.054+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795802ae00 con 0x7f794c0039b0 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.054+0000 7f7966000640 1 -- 172.21.15.123:0/1410142608 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795800b600 con 0x7f794c0039b0 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 -- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 msgr2=0x7f794c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 0x7f794c003e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f795800ba30 tx=0x7f794c005830 comp rx=0 tx=0).stop 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 -- 172.21.15.123:0/1410142608 shutdown_connections 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 --2- 172.21.15.123:0/1410142608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c0039b0 0x7f794c003e40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 -- 172.21.15.123:0/1410142608 >> 172.21.15.123:0/1410142608 conn(0x7f7968073200 msgr2=0x7f7968073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 -- 172.21.15.123:0/1410142608 shutdown_connections 2024-04-25T19:08:18.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.055+0000 7f7967e00640 1 -- 172.21.15.123:0/1410142608 wait complete. 2024-04-25T19:08:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.058+0000 7f7967e00640 1 Processor -- start 2024-04-25T19:08:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.058+0000 7f7967e00640 1 -- start start 2024-04-25T19:08:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c0039b0 0x7f7968143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7968163150 0x7f7968142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7968145f80 con 0x7f7968162800 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79680b10d0 con 0x7f794c0039b0 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f796806d580 con 0x7f7968163150 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41054/0 (socket says 172.21.15.123:41054) 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 -- 172.21.15.123:0/697297835 learned_addr learned my addr 172.21.15.123:0/697297835 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967400640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7968163150 0x7f7968142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7966a00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c0039b0 0x7f7968143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 -- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7968163150 msgr2=0x7f7968142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7968163150 0x7f7968142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 -- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c0039b0 msgr2=0x7f7968143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c0039b0 0x7f7968143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 -- 172.21.15.123:0/697297835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7968143bb0 con 0x7f7968162800 2024-04-25T19:08:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795e000640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f795000c730 tx=0x7f795000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950018070 con 0x7f7968162800 2024-04-25T19:08:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79500072d0 con 0x7f7968162800 2024-04-25T19:08:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500047e0 con 0x7f7968162800 2024-04-25T19:08:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7968163560 con 0x7f7968162800 2024-04-25T19:08:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.061+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7968161d20 con 0x7f7968162800 2024-04-25T19:08:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.063+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968050740 con 0x7f7968162800 2024-04-25T19:08:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.064+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7950004980 con 0x7f7968162800 2024-04-25T19:08:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.065+0000 7f795f400640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 0x7f79440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.066+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f79500948a0 con 0x7f7968162800 2024-04-25T19:08:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.066+0000 7f7966a00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 0x7f79440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.067+0000 7f7966a00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 0x7f79440769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f795800ba00 tx=0x7f7958030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.071+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7968143bb0 con 0x7f7968162800 2024-04-25T19:08:18.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.272+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7968161ad0 con 0x7f7968162800 2024-04-25T19:08:18.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.274+0000 7f795f400640 1 -- 172.21.15.123:0/697297835 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17120 (secure 0 0 0) 0x7f796806ded0 con 0x7f7968162800 2024-04-25T19:08:18.277 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:18.277 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:05:20.272311+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.17","mappings":[{"from":1,"to":2}]},{"pgid":"6.2b","mappings":[{"from":4,"to":7}]},{"pgid":"6.2c","mappings":[{"from":3,"to":5}]},{"pgid":"6.3f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:08:18.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 msgr2=0x7f79440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 0x7f79440769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f795800ba00 tx=0x7f7958030000 comp rx=0 tx=0).stop 2024-04-25T19:08:18.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 msgr2=0x7f7968162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f795000c730 tx=0x7f795000cb30 comp rx=0 tx=0).stop 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 shutdown_connections 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79440744b0 0x7f79440769c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7968163150 0x7f7968142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7968162800 0x7f7968162c10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 --2- 172.21.15.123:0/697297835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f794c0039b0 0x7f7968143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.277+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 >> 172.21.15.123:0/697297835 conn(0x7f7968073200 msgr2=0x7f7968149040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.278+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 shutdown_connections 2024-04-25T19:08:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.278+0000 7f7967e00640 1 -- 172.21.15.123:0/697297835 wait complete. 2024-04-25T19:08:18.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:04:21.554307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:04:22.522839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:08:18.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:08:18.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.583+0000 7ff65fe00640 1 Processor -- start 2024-04-25T19:08:18.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.583+0000 7ff65fe00640 1 -- start start 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff6600ce1c0 0x7ff6600b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff660160290 0x7ff6600b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6600ced20 con 0x7ff6600ce830 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff6600cdd30 con 0x7ff6600ce1c0 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65fe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600cf9d0 con 0x7ff660160290 2024-04-25T19:08:18.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff6600ce830 0x7ff6600b39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42450/0 (socket says 172.21.15.123:42450) 2024-04-25T19:08:18.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.585+0000 7ff65ea00640 1 -- 172.21.15.123:0/2287099015 learned_addr learned my addr 172.21.15.123:0/2287099015 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:18.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff656000640 1 --2- 172.21.15.123:0/2287099015 >> v2:172.21.15.123:3300/0 conn(0x7ff6600ce1c0 0x7ff6600b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff65f400640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff660160290 0x7ff6600b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff65e000640 1 -- 172.21.15.123:0/2287099015 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45685172 0 0) 0x7ff6600ced20 con 0x7ff6600ce830 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff65e000640 1 -- 172.21.15.123:0/2287099015 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff644003520 con 0x7ff6600ce830 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff656000640 1 -- 172.21.15.123:0/2287099015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff660160290 msgr2=0x7ff6600b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff656000640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff660160290 0x7ff6600b79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff656000640 1 -- 172.21.15.123:0/2287099015 >> v1:172.21.15.123:6789/0 conn(0x7ff6600ce830 legacy=0x7ff6600b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.586+0000 7ff656000640 1 -- 172.21.15.123:0/2287099015 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600cfc20 con 0x7ff6600ce1c0 2024-04-25T19:08:18.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.587+0000 7ff656000640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b7470 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff6480097a0 tx=0x7ff64802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e410b6307ca1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.588+0000 7ff65e000640 1 -- 172.21.15.123:0/2287099015 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648037070 con 0x7ff6600ce1c0 2024-04-25T19:08:18.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.588+0000 7ff65e000640 1 -- 172.21.15.123:0/2287099015 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff648032440 con 0x7ff6600ce1c0 2024-04-25T19:08:18.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.588+0000 7ff65e000640 1 -- 172.21.15.123:0/2287099015 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648032810 con 0x7ff6600ce1c0 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.588+0000 7ff65fe00640 1 -- 172.21.15.123:0/2287099015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6600ce1c0 msgr2=0x7ff6600b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.588+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b7470 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff6480097a0 tx=0x7ff64802a180 comp rx=0 tx=0).stop 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 -- 172.21.15.123:0/2287099015 shutdown_connections 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff660160290 0x7ff6600b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2287099015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b7470 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 -- 172.21.15.123:0/2287099015 >> 172.21.15.123:0/2287099015 conn(0x7ff66009b4a0 msgr2=0x7ff66009b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 -- 172.21.15.123:0/2287099015 shutdown_connections 2024-04-25T19:08:18.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.589+0000 7ff65fe00640 1 -- 172.21.15.123:0/2287099015 wait complete. 2024-04-25T19:08:18.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.592+0000 7ff65fe00640 1 Processor -- start 2024-04-25T19:08:18.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.592+0000 7ff65fe00640 1 -- start start 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6600ce830 0x7ff6601f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600ccea0 con 0x7ff660160290 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600ccd20 con 0x7ff6600ce1c0 2024-04-25T19:08:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65fe00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600cd020 con 0x7ff6600ce830 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41062/0 (socket says 172.21.15.123:41062) 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.594+0000 7ff65f400640 1 -- 172.21.15.123:0/2815598061 learned_addr learned my addr 172.21.15.123:0/2815598061 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65ea00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff656000640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6600ce830 0x7ff6601f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 -- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6600ce830 msgr2=0x7ff6601f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6600ce830 0x7ff6601f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 -- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6600ce1c0 msgr2=0x7ff6600b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 -- 172.21.15.123:0/2815598061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6601616f0 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.595+0000 7ff65f400640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff65800d510 tx=0x7ff65800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.596+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580125f0 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.596+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff658019070 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.596+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580129f0 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.596+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601619d0 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.596+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff660161f90 con 0x7ff660160290 2024-04-25T19:08:18.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.597+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66009dce0 con 0x7ff660160290 2024-04-25T19:08:18.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.598+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff658012b90 con 0x7ff660160290 2024-04-25T19:08:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.599+0000 7ff657400640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 0x7ff63c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.599+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7ff658095800 con 0x7ff660160290 2024-04-25T19:08:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.599+0000 7ff65ea00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 0x7ff63c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.600+0000 7ff65ea00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 0x7ff63c0769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff650005c00 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.604+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff65805fd80 con 0x7ff660160290 2024-04-25T19:08:18.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.803+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff660050740 con 0x7ff660160290 2024-04-25T19:08:18.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.804+0000 7ff657400640 1 -- 172.21.15.123:0/2815598061 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7ff65805f680 con 0x7ff660160290 2024-04-25T19:08:18.806 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 msgr2=0x7ff63c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 0x7ff63c0769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff650005c00 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 msgr2=0x7ff66009d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff65800d510 tx=0x7ff65800d9d0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 shutdown_connections 2024-04-25T19:08:18.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff63c0744b0 0x7ff63c0769c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff660160290 0x7ff66009d470 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6600ce830 0x7ff6601f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 --2- 172.21.15.123:0/2815598061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6600ce1c0 0x7ff6600b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 >> 172.21.15.123:0/2815598061 conn(0x7ff66009b4a0 msgr2=0x7ff66015ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 shutdown_connections 2024-04-25T19:08:18.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:18.807+0000 7ff65fe00640 1 -- 172.21.15.123:0/2815598061 wait complete. 2024-04-25T19:08:18.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:08:19.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.113+0000 7f0f99a00640 1 Processor -- start 2024-04-25T19:08:19.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.113+0000 7f0f99a00640 1 -- start start 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f99a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f94070830 0x7f0f940d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f99a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f9415e360 0x7f0f940d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f99a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f940d15d0 con 0x7f0f940cf3d0 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f99a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0f940ce440 con 0x7f0f9415e360 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f99a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0f940d1870 con 0x7f0f94070830 2024-04-25T19:08:19.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f92a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0f940cf3d0 0x7f0f94070120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42458/0 (socket says 172.21.15.123:42458) 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f92a00640 1 -- 172.21.15.123:0/2528241297 learned_addr learned my addr 172.21.15.123:0/2528241297 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.115+0000 7f0f93400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f9415e360 0x7f0f940d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.116+0000 7f0f92000640 1 --2- 172.21.15.123:0/2528241297 >> v2:172.21.15.146:3300/0 conn(0x7f0f94070830 0x7f0f940d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.116+0000 7f0f93400640 1 -- 172.21.15.123:0/2528241297 >> v2:172.21.15.146:3300/0 conn(0x7f0f94070830 msgr2=0x7f0f940d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.116+0000 7f0f93400640 1 --2- 172.21.15.123:0/2528241297 >> v2:172.21.15.146:3300/0 conn(0x7f0f94070830 0x7f0f940d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.116+0000 7f0f93400640 1 -- 172.21.15.123:0/2528241297 >> v1:172.21.15.123:6789/0 conn(0x7f0f940cf3d0 legacy=0x7f0f94070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.116+0000 7f0f93400640 1 -- 172.21.15.123:0/2528241297 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f940d1b50 con 0x7f0f9415e360 2024-04-25T19:08:19.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.117+0000 7f0f93400640 1 --2- 172.21.15.123:0/2528241297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9415e360 0x7f0f940d25b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0f940754e0 tx=0x7f0f8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e12579fd428d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.117+0000 7f0f91600640 1 -- 172.21.15.123:0/2528241297 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84007740 con 0x7f0f9415e360 2024-04-25T19:08:19.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.118+0000 7f0f91600640 1 -- 172.21.15.123:0/2528241297 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f840078e0 con 0x7f0f9415e360 2024-04-25T19:08:19.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.118+0000 7f0f91600640 1 -- 172.21.15.123:0/2528241297 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84007cb0 con 0x7f0f9415e360 2024-04-25T19:08:19.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.118+0000 7f0f99a00640 1 -- 172.21.15.123:0/2528241297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9415e360 msgr2=0x7f0f940d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.118+0000 7f0f99a00640 1 --2- 172.21.15.123:0/2528241297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9415e360 0x7f0f940d25b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0f940754e0 tx=0x7f0f8402a680 comp rx=0 tx=0).stop 2024-04-25T19:08:19.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 -- 172.21.15.123:0/2528241297 shutdown_connections 2024-04-25T19:08:19.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 --2- 172.21.15.123:0/2528241297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9415e360 0x7f0f940d25b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 --2- 172.21.15.123:0/2528241297 >> v2:172.21.15.146:3300/0 conn(0x7f0f94070830 0x7f0f940d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 -- 172.21.15.123:0/2528241297 >> 172.21.15.123:0/2528241297 conn(0x7f0f940d3960 msgr2=0x7f0f940d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:19.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 -- 172.21.15.123:0/2528241297 shutdown_connections 2024-04-25T19:08:19.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.119+0000 7f0f99a00640 1 -- 172.21.15.123:0/2528241297 wait complete. 2024-04-25T19:08:19.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.121+0000 7f0f99a00640 1 Processor -- start 2024-04-25T19:08:19.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.121+0000 7f0f99a00640 1 -- start start 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f940cf3d0 0x7f0f940cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 0x7f0f940ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94070eb0 con 0x7f0f9414bbc0 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94070d30 con 0x7f0f940cf3d0 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f99a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f940cdfb0 con 0x7f0f9415e360 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f93400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f93400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41092/0 (socket says 172.21.15.123:41092) 2024-04-25T19:08:19.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f93400640 1 -- 172.21.15.123:0/3210091095 learned_addr learned my addr 172.21.15.123:0/3210091095 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f92a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f940cf3d0 0x7f0f940cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f92000640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 0x7f0f940ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.123+0000 7f0f93400640 1 -- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 msgr2=0x7f0f940ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f93400640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 0x7f0f940ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f93400640 1 -- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f940cf3d0 msgr2=0x7f0f940cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f93400640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f940cf3d0 0x7f0f940cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f93400640 1 -- 172.21.15.123:0/3210091095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f940c7470 con 0x7f0f9414bbc0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f92000640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 0x7f0f940ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f93400640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402ac70 tx=0x7f0f84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8403f070 con 0x7f0f9414bbc0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f84031bc0 con 0x7f0f9414bbc0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f940c7750 con 0x7f0f9414bbc0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84044460 con 0x7f0f9414bbc0 2024-04-25T19:08:19.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.124+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f940c7d10 con 0x7f0f9414bbc0 2024-04-25T19:08:19.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.125+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f94076a40 con 0x7f0f9414bbc0 2024-04-25T19:08:19.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.127+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f84007790 con 0x7f0f9414bbc0 2024-04-25T19:08:19.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.128+0000 7f0f8aa00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 0x7f0f740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.128+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f0f840b6e00 con 0x7f0f9414bbc0 2024-04-25T19:08:19.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.128+0000 7f0f92a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 0x7f0f740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.129+0000 7f0f92a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 0x7f0f740769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000f60 tx=0x7f0f8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.133+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f84080d30 con 0x7f0f9414bbc0 2024-04-25T19:08:19.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.332+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f940cd820 con 0x7f0f9414bbc0 2024-04-25T19:08:19.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.333+0000 7f0f8aa00640 1 -- 172.21.15.123:0/3210091095 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f0f84085030 con 0x7f0f9414bbc0 2024-04-25T19:08:19.335 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:08:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 msgr2=0x7f0f740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 0x7f0f740769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000f60 tx=0x7f0f8c009040 comp rx=0 tx=0).stop 2024-04-25T19:08:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 msgr2=0x7f0f94157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402ac70 tx=0x7f0f84004a40 comp rx=0 tx=0).stop 2024-04-25T19:08:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 shutdown_connections 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f0f740744b0 0x7f0f740769c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f9414bbc0 0x7f0f94157970 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f9415e360 0x7f0f940ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 --2- 172.21.15.123:0/3210091095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f940cf3d0 0x7f0f940cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 >> 172.21.15.123:0/3210091095 conn(0x7f0f940d3960 msgr2=0x7f0f940b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 shutdown_connections 2024-04-25T19:08:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.336+0000 7f0f99a00640 1 -- 172.21.15.123:0/3210091095 wait complete. 2024-04-25T19:08:19.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:08:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.633+0000 7fd5d9400640 1 Processor -- start 2024-04-25T19:08:19.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.634+0000 7fd5d9400640 1 -- start start 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d9400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5d40cfc10 0x7fd5d40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d9400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5d40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d9400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d4070a20 con 0x7fd5d40b08c0 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d9400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d406c780 con 0x7fd5d40cfc10 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d9400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d406ca60 con 0x7fd5d406fcd0 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd5d40b08c0 0x7fd5d40cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42474/0 (socket says 172.21.15.123:42474) 2024-04-25T19:08:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.636+0000 7fd5d3e00640 1 -- 172.21.15.123:0/1641956910 learned_addr learned my addr 172.21.15.123:0/1641956910 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5cb400640 1 --2- 172.21.15.123:0/1641956910 >> v2:172.21.15.146:3300/0 conn(0x7fd5d40cfc10 0x7fd5d40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d8a00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5d40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582609877 0 0) 0x7fd5d4070a20 con 0x7fd5d40b08c0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b8003520 con 0x7fd5d40b08c0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 968042134 0 0) 0x7fd5b8003520 con 0x7fd5d40b08c0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 msgr2=0x7fd5d40700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5d40700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 >> v2:172.21.15.146:3300/0 conn(0x7fd5d40cfc10 msgr2=0x7fd5d40cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 --2- 172.21.15.123:0/1641956910 >> v2:172.21.15.146:3300/0 conn(0x7fd5d40cfc10 0x7fd5d40cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d406cd40 con 0x7fd5d40b08c0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fd5c40019f0 con 0x7fd5d40b08c0 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 >> v1:172.21.15.123:6789/0 conn(0x7fd5d40b08c0 legacy=0x7fd5d40cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b80039b0 0x7fd5b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b8004d40 0x7fd5b8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d406c780 con 0x7fd5b8004d40 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d406ca60 con 0x7fd5b80039b0 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.637+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b800b9c0 con 0x7fd5d406fcd0 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b8004d40 0x7fd5b8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b80039b0 0x7fd5b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5cb400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 -- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 msgr2=0x7fd5b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d406fcd0 0x7fd5b8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 -- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b80039b0 msgr2=0x7fd5b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b80039b0 0x7fd5b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d8a00640 1 -- 172.21.15.123:0/1641956910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b8003520 con 0x7fd5b8004d40 2024-04-25T19:08:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.638+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b80039b0 0x7fd5b8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:19.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.639+0000 7fd5d8a00640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b8004d40 0x7fd5b8005150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009a90 tx=0x7fd5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c748caedee41d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc036070 con 0x7fd5b8004d40 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc031400 con 0x7fd5b8004d40 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d3400640 1 -- 172.21.15.123:0/1641956910 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc009690 con 0x7fd5b8004d40 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 -- 172.21.15.123:0/1641956910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b8004d40 msgr2=0x7fd5b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 --2- 172.21.15.123:0/1641956910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b8004d40 0x7fd5b8005150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009a90 tx=0x7fd5b8005830 comp rx=0 tx=0).stop 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d8a00640 1 -- 172.21.15.123:0/1641956910 reap_dead start 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 -- 172.21.15.123:0/1641956910 shutdown_connections 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 -- 172.21.15.123:0/1641956910 >> 172.21.15.123:0/1641956910 conn(0x7fd5d40d19f0 msgr2=0x7fd5d40d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 -- 172.21.15.123:0/1641956910 shutdown_connections 2024-04-25T19:08:19.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.640+0000 7fd5d9400640 1 -- 172.21.15.123:0/1641956910 wait complete. 2024-04-25T19:08:19.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.643+0000 7fd5d9400640 1 Processor -- start 2024-04-25T19:08:19.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.643+0000 7fd5d9400640 1 -- start start 2024-04-25T19:08:19.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.644+0000 7fd5d9400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 0x7fd5d4144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d9400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d4141500 0x7fd5d40b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d40704b0 con 0x7fd5d41410f0 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d9400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4070330 con 0x7fd5b8004d40 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d9400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d4070630 con 0x7fd5d4141500 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41108/0 (socket says 172.21.15.123:41108) 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 -- 172.21.15.123:0/976800017 learned_addr learned my addr 172.21.15.123:0/976800017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 0x7fd5d4144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 -- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d4141500 msgr2=0x7fd5d40b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d4141500 0x7fd5d40b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 -- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 msgr2=0x7fd5d4144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 0x7fd5d4144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5cb400640 1 -- 172.21.15.123:0/976800017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d4145400 con 0x7fd5d41410f0 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.645+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 0x7fd5d4144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5cb400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc010860 tx=0x7fd5bc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00f040 con 0x7fd5d41410f0 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00ea80 con 0x7fd5d41410f0 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00ee50 con 0x7fd5d41410f0 2024-04-25T19:08:19.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41568b0 con 0x7fd5d41410f0 2024-04-25T19:08:19.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.646+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d4156e70 con 0x7fd5d41410f0 2024-04-25T19:08:19.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.647+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d41e8090 con 0x7fd5d41410f0 2024-04-25T19:08:19.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.648+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd5bc01c400 con 0x7fd5d41410f0 2024-04-25T19:08:19.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.649+0000 7fd5d0c00640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 0x7fd5b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:19.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.649+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7fd5bc095b20 con 0x7fd5d41410f0 2024-04-25T19:08:19.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.650+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 0x7fd5b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:19.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.650+0000 7fd5d3e00640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 0x7fd5b00769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4071530 tx=0x7fd5c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:19.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.654+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5bc018030 con 0x7fd5d41410f0 2024-04-25T19:08:19.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.857+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5d4050740 con 0x7fd5d41410f0 2024-04-25T19:08:19.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.858+0000 7fd5d0c00640 1 -- 172.21.15.123:0/976800017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7fd5bc060010 con 0x7fd5d41410f0 2024-04-25T19:08:19.860 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.860+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 msgr2=0x7fd5b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.860+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 0x7fd5b00769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4071530 tx=0x7fd5c4009040 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 msgr2=0x7fd5d41e7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc010860 tx=0x7fd5bc010d20 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 shutdown_connections 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd5b00744b0 0x7fd5b00769c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d4141500 0x7fd5d40b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41410f0 0x7fd5d41e7660 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 --2- 172.21.15.123:0/976800017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8004d40 0x7fd5d4144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 >> 172.21.15.123:0/976800017 conn(0x7fd5d40d19f0 msgr2=0x7fd5d40cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 shutdown_connections 2024-04-25T19:08:19.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:19.861+0000 7fd5d9400640 1 -- 172.21.15.123:0/976800017 wait complete. 2024-04-25T19:08:19.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:08:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.152+0000 7f8125600640 1 Processor -- start 2024-04-25T19:08:20.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.153+0000 7f8125600640 1 -- start start 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8125600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 0x7f812006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8125600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f812015ffe0 0x7f81200d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8125600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81200d2840 con 0x7f81200d1a10 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8125600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f81200d1800 con 0x7f812015ffe0 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8125600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81200ce880 con 0x7f81200b2bb0 2024-04-25T19:08:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8124c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f81200d1a10 0x7f81200d1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42480/0 (socket says 172.21.15.123:42480) 2024-04-25T19:08:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.155+0000 7f8124c00640 1 -- 172.21.15.123:0/2979557436 learned_addr learned my addr 172.21.15.123:0/2979557436 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811f400640 1 --2- 172.21.15.123:0/2979557436 >> v2:172.21.15.146:3300/0 conn(0x7f812015ffe0 0x7f81200d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811fe00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 0x7f812006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020066871 0 0) 0x7f81200d2840 con 0x7f81200d1a10 2024-04-25T19:08:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8104003520 con 0x7f81200d1a10 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1282927410 0 0) 0x7f8104003520 con 0x7f81200d1a10 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 msgr2=0x7f812006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 0x7f812006df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 >> v2:172.21.15.146:3300/0 conn(0x7f812015ffe0 msgr2=0x7f81200d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 --2- 172.21.15.123:0/2979557436 >> v2:172.21.15.146:3300/0 conn(0x7f812015ffe0 0x7f81200d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8120071fa0 con 0x7f81200d1a10 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f81180019f0 con 0x7f81200d1a10 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.156+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 >> v1:172.21.15.123:6789/0 conn(0x7f81200d1a10 legacy=0x7f81200d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 0x7f81040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f8104004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8104004d40 0x7f8104005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81200ce880 con 0x7f8104003f80 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f810400ba70 con 0x7f8104004d40 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f810400bbf0 con 0x7f81200b2bb0 2024-04-25T19:08:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f8104004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f8124c00640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8104004d40 0x7f8104005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 -- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 msgr2=0x7f81040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81200b2bb0 0x7f81040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 -- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8104004d40 msgr2=0x7f8104005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8104004d40 0x7f8104005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.157+0000 7f811f400640 1 -- 172.21.15.123:0/2979557436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8104003520 con 0x7f8104003f80 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.158+0000 7f811f400640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f8104004800 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f81200d47f0 tx=0x7f8104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67dccac001cec48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c036070 con 0x7f8104003f80 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c031400 con 0x7f8104003f80 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f811ea00640 1 -- 172.21.15.123:0/2979557436 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c009690 con 0x7f8104003f80 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f8125600640 1 -- 172.21.15.123:0/2979557436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 msgr2=0x7f8104004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f8125600640 1 --2- 172.21.15.123:0/2979557436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f8104004800 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f81200d47f0 tx=0x7f8104005830 comp rx=0 tx=0).stop 2024-04-25T19:08:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f8124c00640 1 -- 172.21.15.123:0/2979557436 reap_dead start 2024-04-25T19:08:20.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f8125600640 1 -- 172.21.15.123:0/2979557436 shutdown_connections 2024-04-25T19:08:20.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.159+0000 7f8125600640 1 -- 172.21.15.123:0/2979557436 >> 172.21.15.123:0/2979557436 conn(0x7f81200d36f0 msgr2=0x7f81200d3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:20.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.160+0000 7f8125600640 1 -- 172.21.15.123:0/2979557436 shutdown_connections 2024-04-25T19:08:20.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.160+0000 7f8125600640 1 -- 172.21.15.123:0/2979557436 wait complete. 2024-04-25T19:08:20.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.162+0000 7f8125600640 1 Processor -- start 2024-04-25T19:08:20.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.162+0000 7f8125600640 1 -- start start 2024-04-25T19:08:20.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81200cc8f0 0x7f8120159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c540 0x7f81200b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81200ce510 con 0x7f8104003f80 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8120072560 con 0x7f81200cc8f0 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f8125600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81201603f0 con 0x7f812015c540 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f811fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f811fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41120/0 (socket says 172.21.15.123:41120) 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f811fe00640 1 -- 172.21.15.123:0/1823025902 learned_addr learned my addr 172.21.15.123:0/1823025902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f811fe00640 1 -- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c540 msgr2=0x7f81200b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.164+0000 7f811fe00640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c540 0x7f81200b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f811fe00640 1 -- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81200cc8f0 msgr2=0x7f8120159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f811fe00640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81200cc8f0 0x7f8120159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f811fe00640 1 -- 172.21.15.123:0/1823025902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812015a390 con 0x7f8104003f80 2024-04-25T19:08:20.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f811fe00640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f811000e7b0 tx=0x7f811000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110017070 con 0x7f8104003f80 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811000c8e0 con 0x7f8104003f80 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81200cd060 con 0x7f8104003f80 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811000ccd0 con 0x7f8104003f80 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.165+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81200b64b0 con 0x7f8104003f80 2024-04-25T19:08:20.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.167+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81200b9670 con 0x7f8104003f80 2024-04-25T19:08:20.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.168+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f811001a070 con 0x7f8104003f80 2024-04-25T19:08:20.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.169+0000 7f8117e00640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 0x7f81000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.169+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f81100952e0 con 0x7f8104003f80 2024-04-25T19:08:20.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.169+0000 7f811f400640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 0x7f81000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.170+0000 7f811f400640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 0x7f81000769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f810c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:20.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.173+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f811005f7d0 con 0x7f8104003f80 2024-04-25T19:08:20.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.372+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f81200ca0f0 con 0x7f8104003f80 2024-04-25T19:08:20.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.373+0000 7f8117e00640 1 -- 172.21.15.123:0/1823025902 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f811005f0d0 con 0x7f8104003f80 2024-04-25T19:08:20.375 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-25T19:08:20.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 msgr2=0x7f81000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 0x7f81000769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f810c02e000 comp rx=0 tx=0).stop 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 msgr2=0x7f81201557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f811000e7b0 tx=0x7f811000ec70 comp rx=0 tx=0).stop 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 shutdown_connections 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f81000744b0 0x7f81000769c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c540 0x7f81200b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81200cc8f0 0x7f8120159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 --2- 172.21.15.123:0/1823025902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8104003f80 0x7f81201557f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 >> 172.21.15.123:0/1823025902 conn(0x7f81200d36f0 msgr2=0x7f8120161510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:20.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 shutdown_connections 2024-04-25T19:08:20.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.376+0000 7f8125600640 1 -- 172.21.15.123:0/1823025902 wait complete. 2024-04-25T19:08:20.403 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T19:08:20.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:20.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T19:08:20.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:20.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:20.493 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T19:08:20.541 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T19:08:20.541 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:08:20.541 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:20.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T19:08:20.599 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T19:08:20.599 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:08:20.599 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:20.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T19:08:20.654 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T19:08:20.654 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:08:20.654 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:20.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T19:08:20.710 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:08:20.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.989+0000 7f4b0f800640 1 Processor -- start 2024-04-25T19:08:20.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.989+0000 7f4b0f800640 1 -- start start 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0f800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4b0806f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0f800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b080b4750 con 0x7f4b080b5f00 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0f800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b080b4a30 con 0x7f4b0806f780 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0f800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0815e310 con 0x7f4b0815df00 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0d800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4b080b5f00 0x7f4b080ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42486/0 (socket says 172.21.15.123:42486) 2024-04-25T19:08:20.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0d800640 1 -- 172.21.15.123:0/1447671761 learned_addr learned my addr 172.21.15.123:0/1447671761 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.992+0000 7f4b0ce00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4b0806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783826244 0 0) 0x7f4b080b4750 con 0x7f4b080b5f00 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aec003520 con 0x7f4b080b5f00 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3288869455 0 0) 0x7f4aec003520 con 0x7f4b080b5f00 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 msgr2=0x7f4b0806f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4b0806f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 >> v1:172.21.15.146:6789/0 conn(0x7f4b0806f780 legacy=0x7f4b080b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080b4d10 con 0x7f4b080b5f00 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f4b00001670 con 0x7f4b080b5f00 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 >> v1:172.21.15.123:6789/0 conn(0x7f4b080b5f00 legacy=0x7f4b080ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4aec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aec003f60 0x7f4aec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4aec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0815e310 con 0x7f4aec004d40 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec00bc00 con 0x7f4aec003f60 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.993+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec00e010 con 0x7f4b0815df00 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4aec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0ce00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4aec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4affe00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aec003f60 0x7f4aec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 -- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 msgr2=0x7f4aec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0815df00 0x7f4aec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 -- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aec003f60 msgr2=0x7f4aec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aec003f60 0x7f4aec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4b0d800640 1 -- 172.21.15.123:0/1447671761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec003520 con 0x7f4aec004d40 2024-04-25T19:08:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.994+0000 7f4affe00640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aec003f60 0x7f4aec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.995+0000 7f4b0d800640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4aec005150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b080b67c0 tx=0x7f4aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10fc77421a2b5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.995+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00007d10 con 0x7f4aec004d40 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.995+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00005ad0 con 0x7f4aec004d40 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.995+0000 7f4b07e00640 1 -- 172.21.15.123:0/1447671761 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00007070 con 0x7f4aec004d40 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 -- 172.21.15.123:0/1447671761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 msgr2=0x7f4aec005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 --2- 172.21.15.123:0/1447671761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4aec005150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b080b67c0 tx=0x7f4aec005830 comp rx=0 tx=0).stop 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0d800640 1 -- 172.21.15.123:0/1447671761 reap_dead start 2024-04-25T19:08:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 -- 172.21.15.123:0/1447671761 shutdown_connections 2024-04-25T19:08:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 -- 172.21.15.123:0/1447671761 >> 172.21.15.123:0/1447671761 conn(0x7f4b080d1ae0 msgr2=0x7f4b080d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 -- 172.21.15.123:0/1447671761 shutdown_connections 2024-04-25T19:08:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.996+0000 7f4b0f800640 1 -- 172.21.15.123:0/1447671761 wait complete. 2024-04-25T19:08:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.999+0000 7f4b0f800640 1 Processor -- start 2024-04-25T19:08:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:20.999+0000 7f4b0f800640 1 -- start start 2024-04-25T19:08:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b081424b0 0x7f4b0806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0809dc30 0x7f4b080700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080aeef0 con 0x7f4aec004d40 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080b5360 con 0x7f4b081424b0 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.000+0000 7f4b0f800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080af070 con 0x7f4b0809dc30 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41130/0 (socket says 172.21.15.123:41130) 2024-04-25T19:08:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 -- 172.21.15.123:0/2402017556 learned_addr learned my addr 172.21.15.123:0/2402017556 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0d800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0809dc30 0x7f4b080700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4affe00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b081424b0 0x7f4b0806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 -- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0809dc30 msgr2=0x7f4b080700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0809dc30 0x7f4b080700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 -- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b081424b0 msgr2=0x7f4b0806fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b081424b0 0x7f4b0806fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.001+0000 7f4b0ce00640 1 -- 172.21.15.123:0/2402017556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081f6410 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b0ce00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4af8007f60 tx=0x7f4af8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af801a020 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8016470 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8016860 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081f5800 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.002+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081f5dc0 con 0x7f4aec004d40 2024-04-25T19:08:21.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.003+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b081521e0 con 0x7f4aec004d40 2024-04-25T19:08:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.005+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4af8005d10 con 0x7f4aec004d40 2024-04-25T19:08:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.005+0000 7f4b05600640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 0x7f4ae40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.006+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f4af8096610 con 0x7f4aec004d40 2024-04-25T19:08:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.006+0000 7f4affe00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 0x7f4ae40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.007+0000 7f4affe00640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 0x7f4ae40769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4af0007fd0 tx=0x7f4af0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:21.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.011+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b081521e0 con 0x7f4aec004d40 2024-04-25T19:08:21.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.253+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b08076a40 con 0x7f4aec004d40 2024-04-25T19:08:21.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.255+0000 7f4b05600640 1 -- 172.21.15.123:0/2402017556 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7f4af8024030 con 0x7f4aec004d40 2024-04-25T19:08:21.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:21.257 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:08:21.257 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:08:21.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 msgr2=0x7f4ae40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 0x7f4ae40769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4af0007fd0 tx=0x7f4af0009040 comp rx=0 tx=0).stop 2024-04-25T19:08:21.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 msgr2=0x7f4b0814f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4af8007f60 tx=0x7f4af8007f90 comp rx=0 tx=0).stop 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 shutdown_connections 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4ae40744b0 0x7f4ae40769c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b0809dc30 0x7f4b080700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b081424b0 0x7f4b0806fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.257+0000 7f4b0f800640 1 --2- 172.21.15.123:0/2402017556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4aec004d40 0x7f4b0814f240 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.258+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 >> 172.21.15.123:0/2402017556 conn(0x7f4b080d1ae0 msgr2=0x7f4b080b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.258+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 shutdown_connections 2024-04-25T19:08:21.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.258+0000 7f4b0f800640 1 -- 172.21.15.123:0/2402017556 wait complete. 2024-04-25T19:08:21.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:08:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.551+0000 7f8c68a00640 1 Processor -- start 2024-04-25T19:08:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.551+0000 7f8c68a00640 1 -- start start 2024-04-25T19:08:21.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 0x7f8c6406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c6406dac0 0x7f8c640b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c640b0af0 0x7f8c6406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8c641462f0 con 0x7f8c6406dac0 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c6406ded0 con 0x7f8c641450d0 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c68a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c641456e0 con 0x7f8c640b0af0 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c5be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c6406dac0 0x7f8c640b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c5be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c6406dac0 0x7f8c640b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41140/0 (socket says 172.21.15.123:41140) 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c5be00640 1 -- 172.21.15.123:0/121987458 learned_addr learned my addr 172.21.15.123:0/121987458 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.554+0000 7f8c63200640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c640b0af0 0x7f8c6406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c5be00640 1 -- 172.21.15.123:0/121987458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c640b0af0 msgr2=0x7f8c6406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:21.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c62800640 1 --2- 172.21.15.123:0/121987458 >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 0x7f8c6406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c5be00640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c640b0af0 0x7f8c6406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c5be00640 1 -- 172.21.15.123:0/121987458 >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 msgr2=0x7f8c6406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c5be00640 1 --2- 172.21.15.123:0/121987458 >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 0x7f8c6406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c5be00640 1 -- 172.21.15.123:0/121987458 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641459c0 con 0x7f8c6406dac0 2024-04-25T19:08:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.555+0000 7f8c62800640 1 --2- 172.21.15.123:0/121987458 >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 0x7f8c6406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.556+0000 7f8c5be00640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640b05b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c64074310 tx=0x7f8c4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d817f961bb8eaa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.556+0000 7f8c61e00640 1 -- 172.21.15.123:0/121987458 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c037070 con 0x7f8c6406dac0 2024-04-25T19:08:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.556+0000 7f8c61e00640 1 -- 172.21.15.123:0/121987458 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c032440 con 0x7f8c6406dac0 2024-04-25T19:08:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.556+0000 7f8c61e00640 1 -- 172.21.15.123:0/121987458 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c032810 con 0x7f8c6406dac0 2024-04-25T19:08:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 -- 172.21.15.123:0/121987458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 msgr2=0x7f8c640b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640b05b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c64074310 tx=0x7f8c4c02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 -- 172.21.15.123:0/121987458 shutdown_connections 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c640b0af0 0x7f8c6406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 --2- 172.21.15.123:0/121987458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640b05b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 --2- 172.21.15.123:0/121987458 >> v2:172.21.15.146:3300/0 conn(0x7f8c641450d0 0x7f8c6406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 -- 172.21.15.123:0/121987458 >> 172.21.15.123:0/121987458 conn(0x7f8c64073200 msgr2=0x7f8c64073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 -- 172.21.15.123:0/121987458 shutdown_connections 2024-04-25T19:08:21.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.557+0000 7f8c68a00640 1 -- 172.21.15.123:0/121987458 wait complete. 2024-04-25T19:08:21.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.560+0000 7f8c68a00640 1 Processor -- start 2024-04-25T19:08:21.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.560+0000 7f8c68a00640 1 -- start start 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 0x7f8c64162c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c641450d0 0x7f8c6415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640b0f00 con 0x7f8c6406dac0 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64145f80 con 0x7f8c640b0af0 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c68a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640b1080 con 0x7f8c641450d0 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41154/0 (socket says 172.21.15.123:41154) 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.562+0000 7f8c62800640 1 -- 172.21.15.123:0/1562973984 learned_addr learned my addr 172.21.15.123:0/1562973984 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:21.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c5be00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 0x7f8c64162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c63200640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c641450d0 0x7f8c6415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 -- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c641450d0 msgr2=0x7f8c6415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c641450d0 0x7f8c6415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 -- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 msgr2=0x7f8c64162c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 0x7f8c64162c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 -- 172.21.15.123:0/1562973984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64163160 con 0x7f8c6406dac0 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c5be00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 0x7f8c64162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:21.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.563+0000 7f8c62800640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c7f0 tx=0x7f8c5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.564+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54015070 con 0x7f8c6406dac0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.564+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c54007820 con 0x7f8c6406dac0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.564+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6413d120 con 0x7f8c6406dac0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.564+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54007bd0 con 0x7f8c6406dac0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.564+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6413d6e0 con 0x7f8c6406dac0 2024-04-25T19:08:21.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.565+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c640af830 con 0x7f8c6406dac0 2024-04-25T19:08:21.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.566+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8c54018070 con 0x7f8c6406dac0 2024-04-25T19:08:21.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.567+0000 7f8c5aa00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 0x7f8c400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:21.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.567+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f8c54095580 con 0x7f8c6406dac0 2024-04-25T19:08:21.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.568+0000 7f8c5be00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 0x7f8c400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:21.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.568+0000 7f8c5be00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 0x7f8c400769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c034000 tx=0x7f8c4c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:21.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.572+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c640af830 con 0x7f8c6406dac0 2024-04-25T19:08:21.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.766+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c6406d8a0 con 0x7f8c6406dac0 2024-04-25T19:08:21.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.767+0000 7f8c5aa00640 1 -- 172.21.15.123:0/1562973984 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17120 (secure 0 0 0) 0x7f8c5405fb10 con 0x7f8c6406dac0 2024-04-25T19:08:21.769 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:21.770 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:05:20.272311+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.17","mappings":[{"from":1,"to":2}]},{"pgid":"6.2b","mappings":[{"from":4,"to":7}]},{"pgid":"6.2c","mappings":[{"from":3,"to":5}]},{"pgid":"6.3f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 msgr2=0x7f8c400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 0x7f8c400769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c034000 tx=0x7f8c4c02e000 comp rx=0 tx=0).stop 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 msgr2=0x7f8c640cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c7f0 tx=0x7f8c5400ccb0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 shutdown_connections 2024-04-25T19:08:21.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f8c400744b0 0x7f8c400769c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c641450d0 0x7f8c6415f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c640b0af0 0x7f8c64162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 --2- 172.21.15.123:0/1562973984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c6406dac0 0x7f8c640cd830 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 >> 172.21.15.123:0/1562973984 conn(0x7f8c64073200 msgr2=0x7f8c640b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 shutdown_connections 2024-04-25T19:08:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:21.770+0000 7f8c68a00640 1 -- 172.21.15.123:0/1562973984 wait complete. 2024-04-25T19:08:21.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:02:26.996878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:04:21.554307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:04:22.522839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:08:21.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:08:22.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.061+0000 7f8413200640 1 Processor -- start 2024-04-25T19:08:22.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.061+0000 7f8413200640 1 -- start start 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8413200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f840c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8413200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f840c06dac0 0x7f840c156890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8413200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f840c06ded0 con 0x7f840c0b0ac0 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8413200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f840c164150 con 0x7f840c06dac0 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8413200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f840c1649f0 con 0x7f840c1645e0 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8411e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f840c0b0ac0 0x7f840c160530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42496/0 (socket says 172.21.15.123:42496) 2024-04-25T19:08:22.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8411e00640 1 -- 172.21.15.123:0/2147518112 learned_addr learned my addr 172.21.15.123:0/2147518112 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8411400640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f840c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.064+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187183179 0 0) 0x7f8400001110 con 0x7f840c0b0ac0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f4003520 con 0x7f840c0b0ac0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8412800640 1 --2- 172.21.15.123:0/2147518112 >> v2:172.21.15.146:3300/0 conn(0x7f840c06dac0 0x7f840c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1639962926 0 0) 0x7f83f4003520 con 0x7f840c0b0ac0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 msgr2=0x7f840c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f840c06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 >> v2:172.21.15.146:3300/0 conn(0x7f840c06dac0 msgr2=0x7f840c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 --2- 172.21.15.123:0/2147518112 >> v2:172.21.15.146:3300/0 conn(0x7f840c06dac0 0x7f840c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c163b20 con 0x7f840c0b0ac0 2024-04-25T19:08:22.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f8400001840 con 0x7f840c0b0ac0 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 >> v1:172.21.15.123:6789/0 conn(0x7f840c0b0ac0 legacy=0x7f840c160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f83f40039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f4003f50 0x7f83f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4004d40 0x7f83f4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f840c164150 con 0x7f83f4003f50 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1649f0 con 0x7f83f4004d40 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83f400bb70 con 0x7f840c1645e0 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.065+0000 7f8411400640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f4003f50 0x7f83f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411e00640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f83f40039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8412800640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4004d40 0x7f83f4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411400640 1 -- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 msgr2=0x7f83f40039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411400640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1645e0 0x7f83f40039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411400640 1 -- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4004d40 msgr2=0x7f83f4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:22.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411400640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4004d40 0x7f83f4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.066+0000 7f8411400640 1 -- 172.21.15.123:0/2147518112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f4003520 con 0x7f83f4003f50 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.067+0000 7f8411400640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f4003f50 0x7f83f4004800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ac0 tx=0x7f83f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0ee33e389a3145 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.067+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc036070 con 0x7f83f4003f50 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc031400 con 0x7f83f4003f50 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8410a00640 1 -- 172.21.15.123:0/2147518112 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc009690 con 0x7f83f4003f50 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8413200640 1 -- 172.21.15.123:0/2147518112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f4003f50 msgr2=0x7f83f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8413200640 1 --2- 172.21.15.123:0/2147518112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f4003f50 0x7f83f4004800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ac0 tx=0x7f83f4005830 comp rx=0 tx=0).stop 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8412800640 1 -- 172.21.15.123:0/2147518112 reap_dead start 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8413200640 1 -- 172.21.15.123:0/2147518112 shutdown_connections 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.068+0000 7f8413200640 1 -- 172.21.15.123:0/2147518112 >> 172.21.15.123:0/2147518112 conn(0x7f840c073090 msgr2=0x7f840c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.069+0000 7f8413200640 1 -- 172.21.15.123:0/2147518112 shutdown_connections 2024-04-25T19:08:22.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.069+0000 7f8413200640 1 -- 172.21.15.123:0/2147518112 wait complete. 2024-04-25T19:08:22.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.071+0000 7f8413200640 1 Processor -- start 2024-04-25T19:08:22.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.071+0000 7f8413200640 1 -- start start 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4003f50 0x7f840c15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1625d0 0x7f840c1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f840c163de0 con 0x7f840c15f7c0 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f840c161340 con 0x7f83f4003f50 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.072+0000 7f8413200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f840c070b20 con 0x7f840c1625d0 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41176/0 (socket says 172.21.15.123:41176) 2024-04-25T19:08:22.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 -- 172.21.15.123:0/1440131494 learned_addr learned my addr 172.21.15.123:0/1440131494 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411e00640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4003f50 0x7f840c15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8412800640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1625d0 0x7f840c1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 -- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1625d0 msgr2=0x7f840c1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1625d0 0x7f840c1629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 -- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4003f50 msgr2=0x7f840c15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4003f50 0x7f840c15e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.073+0000 7f8411400640 1 -- 172.21.15.123:0/1440131494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c0cf4d0 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8411400640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ac0 tx=0x7f83fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc040020 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc003a40 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c162f20 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc036040 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.074+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c15ece0 con 0x7f840c15f7c0 2024-04-25T19:08:22.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.075+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840c050740 con 0x7f840c15f7c0 2024-04-25T19:08:22.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.077+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83fc043020 con 0x7f840c15f7c0 2024-04-25T19:08:22.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.078+0000 7f8406000640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 0x7f83f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.078+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f83fc0b5a40 con 0x7f840c15f7c0 2024-04-25T19:08:22.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.078+0000 7f8411e00640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 0x7f83f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.079+0000 7f8411e00640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 0x7f83f00769f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84000012f0 tx=0x7f8400001580 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.082+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83fc04a030 con 0x7f840c15f7c0 2024-04-25T19:08:22.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.265+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f840c070f30 con 0x7f840c15f7c0 2024-04-25T19:08:22.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.267+0000 7f8406000640 1 -- 172.21.15.123:0/1440131494 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f83fc07ff30 con 0x7f840c15f7c0 2024-04-25T19:08:22.269 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.269+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 msgr2=0x7f83f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.269+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 0x7f83f00769f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84000012f0 tx=0x7f8400001580 comp rx=0 tx=0).stop 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.269+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 msgr2=0x7f840c14d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.269+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ac0 tx=0x7f83fc004730 comp rx=0 tx=0).stop 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 shutdown_connections 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f83f00744e0 0x7f83f00769f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f840c1625d0 0x7f840c1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f840c15f7c0 0x7f840c14d0c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 --2- 172.21.15.123:0/1440131494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f4003f50 0x7f840c15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 >> 172.21.15.123:0/1440131494 conn(0x7f840c073090 msgr2=0x7f840c0af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:22.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 shutdown_connections 2024-04-25T19:08:22.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.270+0000 7f8413200640 1 -- 172.21.15.123:0/1440131494 wait complete. 2024-04-25T19:08:22.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:08:22.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.554+0000 7f471ea00640 1 Processor -- start 2024-04-25T19:08:22.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.554+0000 7f471ea00640 1 -- start start 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.556+0000 7f471ea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180d19e0 0x7f47180d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471ea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47180afbb0 0x7f47180affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471ea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47180ceec0 con 0x7f47180ce9c0 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471ea00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f47180af440 con 0x7f47180afbb0 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471ea00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47180cf1a0 con 0x7f47180d19e0 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47180afbb0 0x7f47180affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471d600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f47180ce9c0 0x7f47180cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42500/0 (socket says 172.21.15.123:42500) 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47180afbb0 0x7f47180affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41178/0 (socket says 172.21.15.123:41178) 2024-04-25T19:08:22.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471d600640 1 -- 172.21.15.123:0/1527146982 learned_addr learned my addr 172.21.15.123:0/1527146982 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.557+0000 7f471cc00640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180d19e0 0x7f47180d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.558+0000 7f471e000640 1 -- 172.21.15.123:0/1527146982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180d19e0 msgr2=0x7f47180d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.558+0000 7f471e000640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180d19e0 0x7f47180d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.558+0000 7f471e000640 1 -- 172.21.15.123:0/1527146982 >> v1:172.21.15.123:6789/0 conn(0x7f47180ce9c0 legacy=0x7f47180cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.558+0000 7f471e000640 1 -- 172.21.15.123:0/1527146982 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47180cf480 con 0x7f47180afbb0 2024-04-25T19:08:22.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.559+0000 7f471e000640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47180affc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104135c6115e2f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.559+0000 7f4717e00640 1 -- 172.21.15.123:0/1527146982 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710007740 con 0x7f47180afbb0 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.559+0000 7f4717e00640 1 -- 172.21.15.123:0/1527146982 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47100078e0 con 0x7f47180afbb0 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.559+0000 7f4717e00640 1 -- 172.21.15.123:0/1527146982 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710007cb0 con 0x7f47180afbb0 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 -- 172.21.15.123:0/1527146982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 msgr2=0x7f47180affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47180affc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471002a560 comp rx=0 tx=0).stop 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 -- 172.21.15.123:0/1527146982 shutdown_connections 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47180affc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 --2- 172.21.15.123:0/1527146982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180d19e0 0x7f47180d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 -- 172.21.15.123:0/1527146982 >> 172.21.15.123:0/1527146982 conn(0x7f471809b4a0 msgr2=0x7f471809b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 -- 172.21.15.123:0/1527146982 shutdown_connections 2024-04-25T19:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.560+0000 7f471ea00640 1 -- 172.21.15.123:0/1527146982 wait complete. 2024-04-25T19:08:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.562+0000 7f471ea00640 1 Processor -- start 2024-04-25T19:08:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.563+0000 7f471ea00640 1 -- start start 2024-04-25T19:08:22.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180ce9c0 0x7f471815ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47180d19e0 0x7f47180cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47180b1ae0 con 0x7f47180afbb0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47180d2330 con 0x7f47180d19e0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471ea00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47180b1c60 con 0x7f47180ce9c0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41194/0 (socket says 172.21.15.123:41194) 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.564+0000 7f471d600640 1 -- 172.21.15.123:0/3902470393 learned_addr learned my addr 172.21.15.123:0/3902470393 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 -- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180ce9c0 msgr2=0x7f471815ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180ce9c0 0x7f471815ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 -- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47180d19e0 msgr2=0x7f47180cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47180d19e0 0x7f47180cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 -- 172.21.15.123:0/3902470393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47180c9350 con 0x7f47180afbb0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471d600640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c550 tx=0x7f470c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c017070 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470c007480 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.565+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47181ea2f0 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.566+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c0059e0 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.566+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47180c6ea0 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.567+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4718050740 con 0x7f47180afbb0 2024-04-25T19:08:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.568+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f470c01a070 con 0x7f47180afbb0 2024-04-25T19:08:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.569+0000 7f4715600640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 0x7f46f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.569+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f470c094870 con 0x7f47180afbb0 2024-04-25T19:08:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.569+0000 7f471cc00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 0x7f46f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.570+0000 7f471cc00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 0x7f46f80769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:22.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.573+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f470c05ee00 con 0x7f47180afbb0 2024-04-25T19:08:22.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.773+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4718071bf0 con 0x7f47180afbb0 2024-04-25T19:08:22.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.774+0000 7f4715600640 1 -- 172.21.15.123:0/3902470393 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f470c05e700 con 0x7f47180afbb0 2024-04-25T19:08:22.776 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-25T19:08:22.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.777+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 msgr2=0x7f46f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.777+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 0x7f46f80769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704008040 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.777+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 msgr2=0x7f47181e9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.777+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c550 tx=0x7f470c00ca10 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.777+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 shutdown_connections 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f46f80744e0 0x7f46f80769f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47180d19e0 0x7f47180cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47180ce9c0 0x7f471815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 --2- 172.21.15.123:0/3902470393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47180afbb0 0x7f47181e9610 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 >> 172.21.15.123:0/3902470393 conn(0x7f471809b4a0 msgr2=0x7f47180b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 shutdown_connections 2024-04-25T19:08:22.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:22.778+0000 7f471ea00640 1 -- 172.21.15.123:0/3902470393 wait complete. 2024-04-25T19:08:22.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:08:23.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.083+0000 7fd302c00640 1 Processor -- start 2024-04-25T19:08:23.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.084+0000 7fd302c00640 1 -- start start 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd302c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2fc15dd70 0x7fd2fc0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd302c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc0c6f50 con 0x7fd2fc0cc430 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd302c00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc0c7230 con 0x7fd2fc15e180 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd302c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0c7b60 con 0x7fd2fc15dd70 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd2fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2fc15dd70 0x7fd2fc0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd300c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd2fc0cc430 0x7fd2fc166d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42512/0 (socket says 172.21.15.123:42512) 2024-04-25T19:08:23.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.086+0000 7fd300c00640 1 -- 172.21.15.123:0/373766285 learned_addr learned my addr 172.21.15.123:0/373766285 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:23.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.087+0000 7fd2fbe00640 1 -- 172.21.15.123:0/373766285 >> v1:172.21.15.146:6789/0 conn(0x7fd2fc15e180 legacy=0x7fd2fc0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:23.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.087+0000 7fd2fbe00640 1 -- 172.21.15.123:0/373766285 >> v1:172.21.15.123:6789/0 conn(0x7fd2fc0cc430 legacy=0x7fd2fc166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.087+0000 7fd2fbe00640 1 -- 172.21.15.123:0/373766285 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0c7510 con 0x7fd2fc15dd70 2024-04-25T19:08:23.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.087+0000 7fd2faa00640 1 -- 172.21.15.123:0/373766285 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040634008 0 0) 0x7fd2fc0c6f50 con 0x7fd2fc0cc430 2024-04-25T19:08:23.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.088+0000 7fd2fbe00640 1 --2- 172.21.15.123:0/373766285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc0c6590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3217eafcb82a1ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.088+0000 7fd2faa00640 1 -- 172.21.15.123:0/373766285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec037070 con 0x7fd2fc15dd70 2024-04-25T19:08:23.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.088+0000 7fd2faa00640 1 -- 172.21.15.123:0/373766285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ec032440 con 0x7fd2fc15dd70 2024-04-25T19:08:23.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.088+0000 7fd2faa00640 1 -- 172.21.15.123:0/373766285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec032810 con 0x7fd2fc15dd70 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 -- 172.21.15.123:0/373766285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 msgr2=0x7fd2fc0c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 --2- 172.21.15.123:0/373766285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc0c6590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 -- 172.21.15.123:0/373766285 shutdown_connections 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 --2- 172.21.15.123:0/373766285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc0c6590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 -- 172.21.15.123:0/373766285 >> 172.21.15.123:0/373766285 conn(0x7fd2fc0ce150 msgr2=0x7fd2fc0ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 -- 172.21.15.123:0/373766285 shutdown_connections 2024-04-25T19:08:23.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.089+0000 7fd302c00640 1 -- 172.21.15.123:0/373766285 wait complete. 2024-04-25T19:08:23.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.091+0000 7fd302c00640 1 Processor -- start 2024-04-25T19:08:23.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.092+0000 7fd302c00640 1 -- start start 2024-04-25T19:08:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e180 0x7fd2fc15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc0af560 0x7fd2fc157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc15d5a0 con 0x7fd2fc15dd70 2024-04-25T19:08:23.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0c7ff0 con 0x7fd2fc0af560 2024-04-25T19:08:23.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd302c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc15d720 con 0x7fd2fc15e180 2024-04-25T19:08:23.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd2fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41216/0 (socket says 172.21.15.123:41216) 2024-04-25T19:08:23.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.093+0000 7fd2fbe00640 1 -- 172.21.15.123:0/3060247972 learned_addr learned my addr 172.21.15.123:0/3060247972 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 -- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e180 msgr2=0x7fd2fc15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e180 0x7fd2fc15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 -- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc0af560 msgr2=0x7fd2fc157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc0af560 0x7fd2fc157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 -- 172.21.15.123:0/3060247972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc15bd90 con 0x7fd2fc15dd70 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2fbe00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.094+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec037040 con 0x7fd2fc15dd70 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.095+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ec004250 con 0x7fd2fc15dd70 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.095+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec032e20 con 0x7fd2fc15dd70 2024-04-25T19:08:23.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.095+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc15c070 con 0x7fd2fc15dd70 2024-04-25T19:08:23.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.095+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc15c630 con 0x7fd2fc15dd70 2024-04-25T19:08:23.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.096+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc076a40 con 0x7fd2fc15dd70 2024-04-25T19:08:23.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.097+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2ec030770 con 0x7fd2fc15dd70 2024-04-25T19:08:23.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.098+0000 7fd2f3e00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 0x7fd2dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.098+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7fd2ec0b6560 con 0x7fd2fc15dd70 2024-04-25T19:08:23.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.098+0000 7fd2fb400640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 0x7fd2dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.099+0000 7fd2fb400640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 0x7fd2dc0769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006fd0 tx=0x7fd2e8006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.103+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2ec080ae0 con 0x7fd2fc15dd70 2024-04-25T19:08:23.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.307+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2fc0cdd80 con 0x7fd2fc15dd70 2024-04-25T19:08:23.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.308+0000 7fd2f3e00640 1 -- 172.21.15.123:0/3060247972 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7fd2ec0803e0 con 0x7fd2fc15dd70 2024-04-25T19:08:23.310 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-25T19:08:23.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 msgr2=0x7fd2dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 0x7fd2dc0769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006fd0 tx=0x7fd2e8006bc0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 msgr2=0x7fd2fc14b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec0046c0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 shutdown_connections 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd2dc0744b0 0x7fd2dc0769c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc0af560 0x7fd2fc157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e180 0x7fd2fc15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 --2- 172.21.15.123:0/3060247972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc15dd70 0x7fd2fc14b5b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 >> 172.21.15.123:0/3060247972 conn(0x7fd2fc0ce150 msgr2=0x7fd2fc160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 shutdown_connections 2024-04-25T19:08:23.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.311+0000 7fd302c00640 1 -- 172.21.15.123:0/3060247972 wait complete. 2024-04-25T19:08:23.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:08:23.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.609+0000 7f79dde00640 1 Processor -- start 2024-04-25T19:08:23.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.610+0000 7f79dde00640 1 -- start start 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79dde00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79dde00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79d8164220 0x7f79d80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79dde00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d80b59c0 con 0x7f79d8163e10 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79dde00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f79d815ab70 con 0x7f79d8164220 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79dde00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f79d80b5ca0 con 0x7f79d80b5400 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79d7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79d8164220 0x7f79d80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79d7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79d8164220 0x7f79d80b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41228/0 (socket says 172.21.15.123:41228) 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.612+0000 7f79d7400640 1 -- 172.21.15.123:0/697043815 learned_addr learned my addr 172.21.15.123:0/697043815 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.613+0000 7f79d6a00640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.613+0000 7f79d7400640 1 -- 172.21.15.123:0/697043815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 msgr2=0x7f79d806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.613+0000 7f79d7400640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.613+0000 7f79d7400640 1 -- 172.21.15.123:0/697043815 >> v1:172.21.15.146:6789/0 conn(0x7f79d8163e10 legacy=0x7f79d80b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:23.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.613+0000 7f79d7400640 1 -- 172.21.15.123:0/697043815 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d80b5f80 con 0x7f79d8164220 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.614+0000 7f79d7400640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d80b0ad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009e50 tx=0x7f79d002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac78b713a0b7b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.614+0000 7f79d5600640 1 -- 172.21.15.123:0/697043815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0036070 con 0x7f79d8164220 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.614+0000 7f79d5600640 1 -- 172.21.15.123:0/697043815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d0031600 con 0x7f79d8164220 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.614+0000 7f79d5600640 1 -- 172.21.15.123:0/697043815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00319d0 con 0x7f79d8164220 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 -- 172.21.15.123:0/697043815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 msgr2=0x7f79d80b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d80b0ad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009e50 tx=0x7f79d002a6c0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 -- 172.21.15.123:0/697043815 shutdown_connections 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d80b0ad0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 --2- 172.21.15.123:0/697043815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 -- 172.21.15.123:0/697043815 >> 172.21.15.123:0/697043815 conn(0x7f79d80d1ae0 msgr2=0x7f79d80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:23.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 -- 172.21.15.123:0/697043815 shutdown_connections 2024-04-25T19:08:23.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.615+0000 7f79dde00640 1 -- 172.21.15.123:0/697043815 wait complete. 2024-04-25T19:08:23.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.618+0000 7f79dde00640 1 Processor -- start 2024-04-25T19:08:23.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.618+0000 7f79dde00640 1 -- start start 2024-04-25T19:08:23.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d80d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 0x7f79d813ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79d8070220 con 0x7f79d8164220 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79d80b6540 con 0x7f79d8163e10 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.619+0000 7f79dde00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f79d80b1010 con 0x7f79d80b5400 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41242/0 (socket says 172.21.15.123:41242) 2024-04-25T19:08:23.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 -- 172.21.15.123:0/2448178448 learned_addr learned my addr 172.21.15.123:0/2448178448 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:23.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d6a00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d80d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d6000640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 0x7f79d813ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 -- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 msgr2=0x7f79d80d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d80d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 -- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 msgr2=0x7f79d813ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 0x7f79d813ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 -- 172.21.15.123:0/2448178448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d80d0dd0 con 0x7f79d8164220 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d6a00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d80d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d6000640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 0x7f79d813ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:23.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.620+0000 7f79d7400640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009e50 tx=0x7f79d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.621+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0041020 con 0x7f79d8164220 2024-04-25T19:08:23.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.621+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d00077a0 con 0x7f79d8164220 2024-04-25T19:08:23.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.621+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d80ceae0 con 0x7f79d8164220 2024-04-25T19:08:23.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.621+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0036040 con 0x7f79d8164220 2024-04-25T19:08:23.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.621+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d80cf0a0 con 0x7f79d8164220 2024-04-25T19:08:23.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.622+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d8076a40 con 0x7f79d8164220 2024-04-25T19:08:23.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.623+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79d0044020 con 0x7f79d8164220 2024-04-25T19:08:23.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.624+0000 7f79cf400640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 0x7f79b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:23.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.624+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f79d00b6470 con 0x7f79d8164220 2024-04-25T19:08:23.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.625+0000 7f79d6a00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 0x7f79b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:23.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.625+0000 7f79d6a00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 0x7f79b80769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006fd0 tx=0x7f79c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:23.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.629+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79d004b020 con 0x7f79d8164220 2024-04-25T19:08:23.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.867+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79d80d2100 con 0x7f79d8164220 2024-04-25T19:08:23.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.868+0000 7f79cf400640 1 -- 172.21.15.123:0/2448178448 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7f79d00809f0 con 0x7f79d8164220 2024-04-25T19:08:23.871 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:23.871 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:08:23.871 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:08:23.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 msgr2=0x7f79b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 0x7f79b80769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006fd0 tx=0x7f79c8008040 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 msgr2=0x7f79d815c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009e50 tx=0x7f79d0004a40 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 shutdown_connections 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f79b80744b0 0x7f79b80769c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79d8164220 0x7f79d815c1d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79d8163e10 0x7f79d813ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 --2- 172.21.15.123:0/2448178448 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79d80b5400 0x7f79d80d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 >> 172.21.15.123:0/2448178448 conn(0x7f79d80d1ae0 msgr2=0x7f79d81460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:23.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.871+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 shutdown_connections 2024-04-25T19:08:23.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:23.872+0000 7f79dde00640 1 -- 172.21.15.123:0/2448178448 wait complete. 2024-04-25T19:08:23.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:08:24.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.162+0000 7fa517a00640 1 Processor -- start 2024-04-25T19:08:24.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.163+0000 7fa517a00640 1 -- start start 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa517a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa510161b10 0x7fa510161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa517a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa510154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa517a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa510071600 con 0x7fa5100d1120 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa517a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa5100d1530 con 0x7fa510161b10 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa517a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa510162460 con 0x7fa510154950 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa507e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa510161b10 0x7fa510161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa507e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa510161b10 0x7fa510161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41258/0 (socket says 172.21.15.123:41258) 2024-04-25T19:08:24.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa507e00640 1 -- 172.21.15.123:0/3129416150 learned_addr learned my addr 172.21.15.123:0/3129416150 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:24.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.165+0000 7fa515000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa5100d1120 0x7fa510070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42520/0 (socket says 172.21.15.123:42520) 2024-04-25T19:08:24.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa507e00640 1 -- 172.21.15.123:0/3129416150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 msgr2=0x7fa510154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:24.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa515a00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa510154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa507e00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa510154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa507e00640 1 -- 172.21.15.123:0/3129416150 >> v1:172.21.15.123:6789/0 conn(0x7fa5100d1120 legacy=0x7fa510070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa507e00640 1 -- 172.21.15.123:0/3129416150 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5100aec30 con 0x7fa510161b10 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.166+0000 7fa50fe00640 1 -- 172.21.15.123:0/3129416150 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586143739 0 0) 0x7fa510071600 con 0x7fa5100d1120 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.167+0000 7fa507e00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa510161b10 0x7fa510161f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009870 tx=0x7fa4f802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf68b442e88535e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.167+0000 7fa50fe00640 1 -- 172.21.15.123:0/3129416150 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8037070 con 0x7fa510161b10 2024-04-25T19:08:24.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.167+0000 7fa50fe00640 1 -- 172.21.15.123:0/3129416150 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f8032440 con 0x7fa510161b10 2024-04-25T19:08:24.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.167+0000 7fa50fe00640 1 -- 172.21.15.123:0/3129416150 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8032810 con 0x7fa510161b10 2024-04-25T19:08:24.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 -- 172.21.15.123:0/3129416150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa510161b10 msgr2=0x7fa510161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa510161b10 0x7fa510161f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009870 tx=0x7fa4f802a130 comp rx=0 tx=0).stop 2024-04-25T19:08:24.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 -- 172.21.15.123:0/3129416150 shutdown_connections 2024-04-25T19:08:24.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa510154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 --2- 172.21.15.123:0/3129416150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa510161b10 0x7fa510161f20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 -- 172.21.15.123:0/3129416150 >> 172.21.15.123:0/3129416150 conn(0x7fa5100d1ae0 msgr2=0x7fa5100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:24.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 -- 172.21.15.123:0/3129416150 shutdown_connections 2024-04-25T19:08:24.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.168+0000 7fa517a00640 1 -- 172.21.15.123:0/3129416150 wait complete. 2024-04-25T19:08:24.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.170+0000 7fa517a00640 1 Processor -- start 2024-04-25T19:08:24.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.171+0000 7fa517a00640 1 -- start start 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa5101459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa510161b10 0x7fa510145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100af4d0 con 0x7fa5100d1120 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100af1f0 con 0x7fa510161b10 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa517a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5101552a0 con 0x7fa510154950 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa515000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa515000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41270/0 (socket says 172.21.15.123:41270) 2024-04-25T19:08:24.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.172+0000 7fa515000640 1 -- 172.21.15.123:0/2618117370 learned_addr learned my addr 172.21.15.123:0/2618117370 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 -- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 msgr2=0x7fa5101459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa5101459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 -- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa510161b10 msgr2=0x7fa510145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa510161b10 0x7fa510145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 -- 172.21.15.123:0/2618117370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5100b6040 con 0x7fa5100d1120 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa515000640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c430 tx=0x7fa50000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5000074e0 con 0x7fa5100d1120 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5000047e0 con 0x7fa5100d1120 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500004bd0 con 0x7fa5100d1120 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5100b5a80 con 0x7fa5100d1120 2024-04-25T19:08:24.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.173+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51015a080 con 0x7fa5100d1120 2024-04-25T19:08:24.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.175+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa510076a40 con 0x7fa5100d1120 2024-04-25T19:08:24.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.176+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa500011070 con 0x7fa5100d1120 2024-04-25T19:08:24.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.177+0000 7fa50d600640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 0x7fa4ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.177+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7fa500094fb0 con 0x7fa5100d1120 2024-04-25T19:08:24.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.177+0000 7fa507e00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 0x7fa4ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.178+0000 7fa507e00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 0x7fa4ec0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034000 tx=0x7fa4f802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.182+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa50005f5d0 con 0x7fa5100d1120 2024-04-25T19:08:24.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:24.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.391+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa51015be40 con 0x7fa5100d1120 2024-04-25T19:08:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.394+0000 7fa50d600640 1 -- 172.21.15.123:0/2618117370 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17120 (secure 0 0 0) 0x7fa50005eed0 con 0x7fa5100d1120 2024-04-25T19:08:24.397 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:24.397 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:05:20.272311+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.17","mappings":[{"from":1,"to":2}]},{"pgid":"6.2b","mappings":[{"from":4,"to":7}]},{"pgid":"6.2c","mappings":[{"from":3,"to":5}]},{"pgid":"6.3f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:08:24.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 msgr2=0x7fa4ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 0x7fa4ec0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034000 tx=0x7fa4f802c000 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 msgr2=0x7fa5100b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c430 tx=0x7fa50000c8f0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 shutdown_connections 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa4ec0744e0 0x7fa4ec0769f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa510161b10 0x7fa510145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa510154950 0x7fa5101459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 --2- 172.21.15.123:0/2618117370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5100d1120 0x7fa5100b5130 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.397+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 >> 172.21.15.123:0/2618117370 conn(0x7fa5100d1ae0 msgr2=0x7fa51015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.398+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 shutdown_connections 2024-04-25T19:08:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.398+0000 7fa517a00640 1 -- 172.21.15.123:0/2618117370 wait complete. 2024-04-25T19:08:24.426 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:08:24.427 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:08:24.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.740+0000 7f7b98000640 1 Processor -- start 2024-04-25T19:08:24.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.740+0000 7f7b98000640 1 -- start start 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b98000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b90161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b98000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b90071600 con 0x7f7b900d1120 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b98000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b90162460 con 0x7f7b90154950 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b98000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7b900d1530 con 0x7f7b90161b10 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b95600640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7b900d1120 0x7f7b90070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:42532/0 (socket says 172.21.15.123:42532) 2024-04-25T19:08:24.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.742+0000 7f7b95600640 1 -- 172.21.15.123:0/50182846 learned_addr learned my addr 172.21.15.123:0/50182846 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018066187 0 0) 0x7f7b90071600 con 0x7f7b900d1120 2024-04-25T19:08:24.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b94c00640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b90161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b74003520 con 0x7f7b900d1120 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3372810604 0 0) 0x7f7b74003520 con 0x7f7b900d1120 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 msgr2=0x7f7b90161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b90161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 >> v1:172.21.15.146:6789/0 conn(0x7f7b90154950 legacy=0x7f7b90154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b900aec30 con 0x7f7b900d1120 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f7b800019f0 con 0x7f7b900d1120 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 >> v1:172.21.15.123:6789/0 conn(0x7f7b900d1120 legacy=0x7f7b90070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b74003fb0 0x7f7b74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b74004d90 0x7f7b74005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900d1530 con 0x7f7b74004d90 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400bcf0 con 0x7f7b74003fb0 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.743+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7400bef0 con 0x7f7b90161b10 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b74004d90 0x7f7b74005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b95600640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 -- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 msgr2=0x7f7b740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b90161b10 0x7f7b740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 -- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b74003fb0 msgr2=0x7f7b74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b74003fb0 0x7f7b74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.744+0000 7f7b96000640 1 -- 172.21.15.123:0/50182846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b74003520 con 0x7f7b74004d90 2024-04-25T19:08:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.745+0000 7f7b96000640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b74004d90 0x7f7b74005240 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001720 tx=0x7f7b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9a5c5d9fe64306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88001370 con 0x7f7b74004d90 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b880052d0 con 0x7f7b74004d90 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b8fe00640 1 -- 172.21.15.123:0/50182846 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8802abc0 con 0x7f7b74004d90 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b98000640 1 -- 172.21.15.123:0/50182846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b74004d90 msgr2=0x7f7b74005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b98000640 1 --2- 172.21.15.123:0/50182846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b74004d90 0x7f7b74005240 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001720 tx=0x7f7b74005830 comp rx=0 tx=0).stop 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.746+0000 7f7b96000640 1 -- 172.21.15.123:0/50182846 reap_dead start 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.747+0000 7f7b98000640 1 -- 172.21.15.123:0/50182846 shutdown_connections 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.747+0000 7f7b98000640 1 -- 172.21.15.123:0/50182846 >> 172.21.15.123:0/50182846 conn(0x7f7b900d1ae0 msgr2=0x7f7b900d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.747+0000 7f7b98000640 1 -- 172.21.15.123:0/50182846 shutdown_connections 2024-04-25T19:08:24.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.747+0000 7f7b98000640 1 -- 172.21.15.123:0/50182846 wait complete. 2024-04-25T19:08:24.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.749+0000 7f7b98000640 1 Processor -- start 2024-04-25T19:08:24.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.749+0000 7f7b98000640 1 -- start start 2024-04-25T19:08:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b74004d90 0x7f7b901f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b90154230 0x7f7b9015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900af4d0 con 0x7f7b9015b450 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900af1f0 con 0x7f7b90154230 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b98000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7b900ae7e0 con 0x7f7b74004d90 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41280/0 (socket says 172.21.15.123:41280) 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b96000640 1 -- 172.21.15.123:0/1594436017 learned_addr learned my addr 172.21.15.123:0/1594436017 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b95600640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b74004d90 0x7f7b901f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b94c00640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b90154230 0x7f7b9015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b96000640 1 -- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b74004d90 msgr2=0x7f7b901f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.751+0000 7f7b96000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b74004d90 0x7f7b901f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b96000640 1 -- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b90154230 msgr2=0x7f7b9015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b96000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b90154230 0x7f7b9015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:24.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b96000640 1 -- 172.21.15.123:0/1594436017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901f78e0 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b96000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001840 tx=0x7f7b88001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88037040 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88040070 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9015b860 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88001370 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.752+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b90159c90 con 0x7f7b9015b450 2024-04-25T19:08:24.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.754+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b900713d0 con 0x7f7b9015b450 2024-04-25T19:08:24.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.755+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7b880052d0 con 0x7f7b9015b450 2024-04-25T19:08:24.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.756+0000 7f7b8d600640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 0x7f7b700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:24.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.756+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f7b880b4e60 con 0x7f7b9015b450 2024-04-25T19:08:24.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.756+0000 7f7b95600640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 0x7f7b700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:24.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.757+0000 7f7b95600640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 0x7f7b700769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b80002230 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:24.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:24.761+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b900713d0 con 0x7f7b9015b450 2024-04-25T19:08:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.001+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b90076a40 con 0x7f7b9015b450 2024-04-25T19:08:25.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.002+0000 7f7b8d600640 1 -- 172.21.15.123:0/1594436017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7f7b9015b860 con 0x7f7b9015b450 2024-04-25T19:08:25.004 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:25.004 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:08:25.005 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:08:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.004+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 msgr2=0x7f7b700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.004+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 0x7f7b700769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b80002230 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 msgr2=0x7f7b901f9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001840 tx=0x7f7b88001120 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 shutdown_connections 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7b700744b0 0x7f7b700769c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b9015b450 0x7f7b901f9f10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b90154230 0x7f7b9015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 --2- 172.21.15.123:0/1594436017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b74004d90 0x7f7b901f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 >> 172.21.15.123:0/1594436017 conn(0x7f7b900d1ae0 msgr2=0x7f7b90158250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 shutdown_connections 2024-04-25T19:08:25.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.005+0000 7f7b98000640 1 -- 172.21.15.123:0/1594436017 wait complete. 2024-04-25T19:08:25.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:08:25.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.298+0000 7efd17e00640 1 Processor -- start 2024-04-25T19:08:25.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.298+0000 7efd17e00640 1 -- start start 2024-04-25T19:08:25.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.300+0000 7efd17e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd1806fe20 0x7efd1815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd17e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd17e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efd1815e2f0 0x7efd1815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd17e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efd1806ded0 con 0x7efd1806fe20 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd17e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efd1815f070 con 0x7efd1815e2f0 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd17e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efd1806f340 con 0x7efd1806d580 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd1806fe20 0x7efd1815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd1806fe20 0x7efd1815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57732/0 (socket says 172.21.15.123:57732) 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 -- 172.21.15.123:0/3473232328 learned_addr learned my addr 172.21.15.123:0/3473232328 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 -- 172.21.15.123:0/3473232328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 msgr2=0x7efd1806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 --2- 172.21.15.123:0/3473232328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 -- 172.21.15.123:0/3473232328 >> v2:172.21.15.146:3300/0 conn(0x7efd1815e2f0 msgr2=0x7efd1815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 --2- 172.21.15.123:0/3473232328 >> v2:172.21.15.146:3300/0 conn(0x7efd1815e2f0 0x7efd1815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.301+0000 7efd16a00640 1 -- 172.21.15.123:0/3473232328 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1806f620 con 0x7efd1806fe20 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.302+0000 7efd16a00640 1 --2- 172.21.15.123:0/3473232328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd1815b6b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd18072df0 tx=0x7efd0802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c15d97aec9c85ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.303+0000 7efd16000640 1 -- 172.21.15.123:0/3473232328 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08037070 con 0x7efd1806fe20 2024-04-25T19:08:25.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.303+0000 7efd16000640 1 -- 172.21.15.123:0/3473232328 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08032440 con 0x7efd1806fe20 2024-04-25T19:08:25.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.303+0000 7efd16000640 1 -- 172.21.15.123:0/3473232328 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08032810 con 0x7efd1806fe20 2024-04-25T19:08:25.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.303+0000 7efd17e00640 1 -- 172.21.15.123:0/3473232328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 msgr2=0x7efd1815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.303+0000 7efd17e00640 1 --2- 172.21.15.123:0/3473232328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd1815b6b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd18072df0 tx=0x7efd0802a140 comp rx=0 tx=0).stop 2024-04-25T19:08:25.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 -- 172.21.15.123:0/3473232328 shutdown_connections 2024-04-25T19:08:25.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 --2- 172.21.15.123:0/3473232328 >> v2:172.21.15.146:3300/0 conn(0x7efd1815e2f0 0x7efd1815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 --2- 172.21.15.123:0/3473232328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd1806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 --2- 172.21.15.123:0/3473232328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd1815b6b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 -- 172.21.15.123:0/3473232328 >> 172.21.15.123:0/3473232328 conn(0x7efd18073200 msgr2=0x7efd18073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:25.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 -- 172.21.15.123:0/3473232328 shutdown_connections 2024-04-25T19:08:25.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.304+0000 7efd17e00640 1 -- 172.21.15.123:0/3473232328 wait complete. 2024-04-25T19:08:25.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.306+0000 7efd17e00640 1 Processor -- start 2024-04-25T19:08:25.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.306+0000 7efd17e00640 1 -- start start 2024-04-25T19:08:25.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd18132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd1815e2f0 0x7efd18141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd1815ec40 con 0x7efd1806fe20 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efd1815bbf0 con 0x7efd1815e2f0 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd17e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efd1815edc0 con 0x7efd1806d580 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd0e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd0e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57734/0 (socket says 172.21.15.123:57734) 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd0e000640 1 -- 172.21.15.123:0/4224976666 learned_addr learned my addr 172.21.15.123:0/4224976666 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:25.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd0e000640 1 -- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 msgr2=0x7efd18132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.308+0000 7efd0e000640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd18132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0e000640 1 -- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd1815e2f0 msgr2=0x7efd18141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0e000640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd1815e2f0 0x7efd18141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0e000640 1 -- 172.21.15.123:0/4224976666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd18145810 con 0x7efd1806fe20 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0e000640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b410 tx=0x7efd0000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00015020 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00011850 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00011c40 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd181ea840 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.309+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd18144a50 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.310+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd181444e0 con 0x7efd1806fe20 2024-04-25T19:08:25.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.312+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd00018020 con 0x7efd1806fe20 2024-04-25T19:08:25.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.313+0000 7efd0f400640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 0x7efcf40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.313+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7efd000953a0 con 0x7efd1806fe20 2024-04-25T19:08:25.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.314+0000 7efd16a00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 0x7efcf40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.314+0000 7efd16a00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 0x7efcf40769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd08034000 tx=0x7efd0802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.318+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd0001f020 con 0x7efd1806fe20 2024-04-25T19:08:25.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.564+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd18050740 con 0x7efd1806fe20 2024-04-25T19:08:25.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.566+0000 7efd0f400640 1 -- 172.21.15.123:0/4224976666 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5190 (secure 0 0 0) 0x7efd0005f920 con 0x7efd1806fe20 2024-04-25T19:08:25.568 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:25.568 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:04:23.546022+0000","modified":"2024-04-25T19:04:26.642728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4801":{"gid":4801,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6835/4124573423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4124573423},{"type":"v1","addr":"172.21.15.123:6835","nonce":4124573423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4801,"qdb_cluster":[4801]},"id":2}]} 2024-04-25T19:08:25.568 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-25T19:08:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 msgr2=0x7efcf40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 0x7efcf40769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd08034000 tx=0x7efd0802c000 comp rx=0 tx=0).stop 2024-04-25T19:08:25.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 msgr2=0x7efd181ea300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b410 tx=0x7efd0000b8d0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 shutdown_connections 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcf40744b0 0x7efcf40769c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd1815e2f0 0x7efd18141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1806fe20 0x7efd181ea300 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 --2- 172.21.15.123:0/4224976666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd1806d580 0x7efd18132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.568+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 >> 172.21.15.123:0/4224976666 conn(0x7efd18073200 msgr2=0x7efd18161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.569+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 shutdown_connections 2024-04-25T19:08:25.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.569+0000 7efd17e00640 1 -- 172.21.15.123:0/4224976666 wait complete. 2024-04-25T19:08:25.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:08:25.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.833+0000 7f173b800640 1 Processor -- start 2024-04-25T19:08:25.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.833+0000 7f173b800640 1 -- start start 2024-04-25T19:08:25.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.835+0000 7f173b800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f1734070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.835+0000 7f173b800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1734161b10 0x7f1734161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.835+0000 7f173b800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1734162460 con 0x7f1734154950 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.835+0000 7f173b800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f17340d1530 con 0x7f1734161b10 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.835+0000 7f173b800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17340aec30 con 0x7f17340d1120 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1739800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1734154950 0x7f1734154d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55484/0 (socket says 172.21.15.123:55484) 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1739800640 1 -- 172.21.15.123:0/4087962151 learned_addr learned my addr 172.21.15.123:0/4087962151 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1738e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f1734070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> v2:172.21.15.146:3300/0 conn(0x7f1734161b10 0x7f1734161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2503275463 0 0) 0x7f172c001490 con 0x7f1734154950 2024-04-25T19:08:25.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f171c003520 con 0x7f1734154950 2024-04-25T19:08:25.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1978876798 0 0) 0x7f171c003520 con 0x7f1734154950 2024-04-25T19:08:25.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.836+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 msgr2=0x7f1734070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f1734070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 >> v2:172.21.15.146:3300/0 conn(0x7f1734161b10 msgr2=0x7f1734161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 --2- 172.21.15.123:0/4087962151 >> v2:172.21.15.146:3300/0 conn(0x7f1734161b10 0x7f1734161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17340aef10 con 0x7f1734154950 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f172c001bc0 con 0x7f1734154950 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1738e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f1734070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 >> v1:172.21.15.123:6789/0 conn(0x7f1734154950 legacy=0x7f1734154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0039b0 0x7f171c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f171c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f171c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17340d1530 con 0x7f171c004800 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17340aec30 con 0x7f171c0039b0 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f171c005560 con 0x7f17340d1120 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> v2:172.21.15.146:3300/0 conn(0x7f1734161b10 0x7f1734161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f171c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.837+0000 7f1738e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0039b0 0x7f171c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1739800640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f171c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1733e00640 1 -- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 msgr2=0x7f171c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f171c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1733e00640 1 -- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0039b0 msgr2=0x7f171c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:25.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f171c0039b0 0x7f171c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1733e00640 1 -- 172.21.15.123:0/4087962151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c003520 con 0x7f171c004800 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.838+0000 7f1739800640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340d1120 0x7f171c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.839+0000 7f1733e00640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f171c004c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1734070730 tx=0x7f171c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16fede6111b4046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.839+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720037070 con 0x7f171c004800 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.839+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172002aca0 con 0x7f171c004800 2024-04-25T19:08:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.839+0000 7f1733400640 1 -- 172.21.15.123:0/4087962151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000f040 con 0x7f171c004800 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 -- 172.21.15.123:0/4087962151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 msgr2=0x7f171c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 --2- 172.21.15.123:0/4087962151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f171c004c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1734070730 tx=0x7f171c005830 comp rx=0 tx=0).stop 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f1739800640 1 -- 172.21.15.123:0/4087962151 reap_dead start 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 -- 172.21.15.123:0/4087962151 shutdown_connections 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 -- 172.21.15.123:0/4087962151 >> 172.21.15.123:0/4087962151 conn(0x7f17340d1ae0 msgr2=0x7f17340d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 -- 172.21.15.123:0/4087962151 shutdown_connections 2024-04-25T19:08:25.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.840+0000 7f173b800640 1 -- 172.21.15.123:0/4087962151 wait complete. 2024-04-25T19:08:25.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.844+0000 7f173b800640 1 Processor -- start 2024-04-25T19:08:25.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.844+0000 7f173b800640 1 -- start start 2024-04-25T19:08:25.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340b5d10 0x7f17340cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173414d0d0 0x7f173414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17340ae7e0 con 0x7f171c004800 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17340af4d0 con 0x7f173414d0d0 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f173b800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17340ae960 con 0x7f17340b5d10 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f1738e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f1738e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57754/0 (socket says 172.21.15.123:57754) 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.846+0000 7f1738e00640 1 -- 172.21.15.123:0/224215116 learned_addr learned my addr 172.21.15.123:0/224215116 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1733e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340b5d10 0x7f17340cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1739800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173414d0d0 0x7f173414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 -- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340b5d10 msgr2=0x7f17340cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340b5d10 0x7f17340cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 -- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173414d0d0 msgr2=0x7f173414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173414d0d0 0x7f173414d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 -- 172.21.15.123:0/224215116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17340b4c70 con 0x7f171c004800 2024-04-25T19:08:25.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1738e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1728010890 tx=0x7f1728010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.847+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172801a070 con 0x7f171c004800 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.848+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172800cb40 con 0x7f171c004800 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.848+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172800f040 con 0x7f171c004800 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.848+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17340b4f50 con 0x7f171c004800 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.848+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17340b5510 con 0x7f171c004800 2024-04-25T19:08:25.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.849+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1734076a40 con 0x7f171c004800 2024-04-25T19:08:25.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.851+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1728015470 con 0x7f171c004800 2024-04-25T19:08:25.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.852+0000 7f1730c00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 0x7f17140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:25.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.852+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f1728095ae0 con 0x7f171c004800 2024-04-25T19:08:25.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.852+0000 7f1733e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 0x7f17140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:25.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.853+0000 7f1733e00640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 0x7f17140769f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1720034000 tx=0x7f172002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:25.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:25.857+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1728060060 con 0x7f171c004800 2024-04-25T19:08:26.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.060+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1734070250 con 0x7f171c004800 2024-04-25T19:08:26.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.063+0000 7f1730c00640 1 -- 172.21.15.123:0/224215116 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17120 (secure 0 0 0) 0x7f172805f960 con 0x7f171c004800 2024-04-25T19:08:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:26.065 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:05:20.272311+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:04:21.554307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T19:04:22.522839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.17","mappings":[{"from":1,"to":2}]},{"pgid":"6.2b","mappings":[{"from":4,"to":7}]},{"pgid":"6.2c","mappings":[{"from":3,"to":5}]},{"pgid":"6.3f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 msgr2=0x7f17140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 0x7f17140769f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1720034000 tx=0x7f172002e000 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 msgr2=0x7f17340cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1728010890 tx=0x7f1728010d50 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 shutdown_connections 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f17140744e0 0x7f17140769f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173414d0d0 0x7f173414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17340b5d10 0x7f17340cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 --2- 172.21.15.123:0/224215116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f171c004800 0x7f17340cf2b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.065+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 >> 172.21.15.123:0/224215116 conn(0x7f17340d1ae0 msgr2=0x7f173415e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.066+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 shutdown_connections 2024-04-25T19:08:26.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.066+0000 7f173b800640 1 -- 172.21.15.123:0/224215116 wait complete. 2024-04-25T19:08:26.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T19:08:26.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.337+0000 7f7a47e00640 1 Processor -- start 2024-04-25T19:08:26.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.338+0000 7f7a47e00640 1 -- start start 2024-04-25T19:08:26.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4806dc90 0x7f7a48142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a48142a70 0x7f7a480b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a480b5c70 con 0x7f7a4813fbf0 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480b5f50 con 0x7f7a48142a70 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480b6230 con 0x7f7a4806dc90 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a47400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a48142a70 0x7f7a480b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a46a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7a4813fbf0 0x7f7a4806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55500/0 (socket says 172.21.15.123:55500) 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a46a00640 1 -- 172.21.15.123:0/2787012943 learned_addr learned my addr 172.21.15.123:0/2787012943 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.340+0000 7f7a46000640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4806dc90 0x7f7a48142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.341+0000 7f7a47400640 1 -- 172.21.15.123:0/2787012943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4806dc90 msgr2=0x7f7a48142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.341+0000 7f7a47400640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4806dc90 0x7f7a48142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.341+0000 7f7a47400640 1 -- 172.21.15.123:0/2787012943 >> v1:172.21.15.123:6789/0 conn(0x7f7a4813fbf0 legacy=0x7f7a4806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.341+0000 7f7a47400640 1 -- 172.21.15.123:0/2787012943 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a480b6510 con 0x7f7a48142a70 2024-04-25T19:08:26.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.341+0000 7f7a45600640 1 -- 172.21.15.123:0/2787012943 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305972440 0 0) 0x7f7a480b5c70 con 0x7f7a4813fbf0 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.342+0000 7f7a47400640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a480b4640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009d90 tx=0x7f7a4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3001ca02cb2e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.342+0000 7f7a45600640 1 -- 172.21.15.123:0/2787012943 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40007740 con 0x7f7a48142a70 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.342+0000 7f7a45600640 1 -- 172.21.15.123:0/2787012943 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a400078e0 con 0x7f7a48142a70 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.342+0000 7f7a45600640 1 -- 172.21.15.123:0/2787012943 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40007cb0 con 0x7f7a48142a70 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 -- 172.21.15.123:0/2787012943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 msgr2=0x7f7a480b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a480b4640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009d90 tx=0x7f7a4002a660 comp rx=0 tx=0).stop 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 -- 172.21.15.123:0/2787012943 shutdown_connections 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a480b4640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 --2- 172.21.15.123:0/2787012943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4806dc90 0x7f7a48142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 -- 172.21.15.123:0/2787012943 >> 172.21.15.123:0/2787012943 conn(0x7f7a48073200 msgr2=0x7f7a48073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 -- 172.21.15.123:0/2787012943 shutdown_connections 2024-04-25T19:08:26.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.343+0000 7f7a47e00640 1 -- 172.21.15.123:0/2787012943 wait complete. 2024-04-25T19:08:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.345+0000 7f7a47e00640 1 Processor -- start 2024-04-25T19:08:26.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.345+0000 7f7a47e00640 1 -- start start 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.346+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4813fbf0 0x7f7a48141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.346+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.346+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a48156bf0 0x7f7a4815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a47e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48143000 con 0x7f7a48142a70 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a47e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48142e80 con 0x7f7a4813fbf0 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a47e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480b4b80 con 0x7f7a48156bf0 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57782/0 (socket says 172.21.15.123:57782) 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 -- 172.21.15.123:0/528349981 learned_addr learned my addr 172.21.15.123:0/528349981 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:26.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 -- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a48156bf0 msgr2=0x7f7a4815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a48156bf0 0x7f7a4815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 -- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4813fbf0 msgr2=0x7f7a48141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4813fbf0 0x7f7a48141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.347+0000 7f7a46000640 1 -- 172.21.15.123:0/528349981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a48160f10 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a46000640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d620 tx=0x7f7a3400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400b620 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34019070 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481611f0 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ba00 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.348+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481563f0 con 0x7f7a48142a70 2024-04-25T19:08:26.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.349+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a48050740 con 0x7f7a48142a70 2024-04-25T19:08:26.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.350+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a3400bba0 con 0x7f7a48142a70 2024-04-25T19:08:26.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.351+0000 7f7a3ea00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 0x7f7a280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.351+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6217+0+0 (secure 0 0 0) 0x7f7a34095870 con 0x7f7a48142a70 2024-04-25T19:08:26.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.352+0000 7f7a46a00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 0x7f7a280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.352+0000 7f7a46a00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 0x7f7a280769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007fd0 tx=0x7f7a38007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.356+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a3405fdf0 con 0x7f7a48142a70 2024-04-25T19:08:26.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.597+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7a48160c60 con 0x7f7a48142a70 2024-04-25T19:08:26.600 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-04-25T19:08:26.599+0000 7fed86c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:08:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.602+0000 7f7a3ea00640 1 -- 172.21.15.123:0/528349981 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f7a3405f6f0 con 0x7f7a48142a70 2024-04-25T19:08:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T19:08:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 msgr2=0x7f7a280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 0x7f7a280769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007fd0 tx=0x7f7a38007960 comp rx=0 tx=0).stop 2024-04-25T19:08:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 msgr2=0x7f7a4814de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d620 tx=0x7f7a3400dae0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 shutdown_connections 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f7a280744b0 0x7f7a280769c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a48156bf0 0x7f7a4815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a48142a70 0x7f7a4814de50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.605+0000 7f7a47e00640 1 --2- 172.21.15.123:0/528349981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4813fbf0 0x7f7a48141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.606+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 >> 172.21.15.123:0/528349981 conn(0x7f7a48073200 msgr2=0x7f7a4806f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.606+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 shutdown_connections 2024-04-25T19:08:26.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.606+0000 7f7a47e00640 1 -- 172.21.15.123:0/528349981 wait complete. 2024-04-25T19:08:26.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T19:08:26.764 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-25T19:08:26.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.888+0000 7ff718400640 1 Processor -- start 2024-04-25T19:08:26.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.888+0000 7ff718400640 1 -- start start 2024-04-25T19:08:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.890+0000 7ff718400640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff710070820 0x7ff7101609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.890+0000 7ff718400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7100cf300 con 0x7ff7100cdca0 2024-04-25T19:08:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.890+0000 7ff718400640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7100cf5e0 con 0x7ff7100d03a0 2024-04-25T19:08:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.890+0000 7ff718400640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff7100cff10 con 0x7ff710070820 2024-04-25T19:08:26.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff716400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff7100cdca0 0x7ff71006f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55512/0 (socket says 172.21.15.123:55512) 2024-04-25T19:08:26.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff716400640 1 -- 172.21.15.123:0/1743199493 learned_addr learned my addr 172.21.15.123:0/1743199493 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1753702200 0 0) 0x7ff7100cf300 con 0x7ff7100cdca0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff715000640 1 --2- 172.21.15.123:0/1743199493 >> v2:172.21.15.146:3300/0 conn(0x7ff710070820 0x7ff7101609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f8003520 con 0x7ff7100cdca0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3933582923 0 0) 0x7ff6f8003520 con 0x7ff7100cdca0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 >> v2:172.21.15.146:3300/0 conn(0x7ff710070820 msgr2=0x7ff7101609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 --2- 172.21.15.123:0/1743199493 >> v2:172.21.15.146:3300/0 conn(0x7ff710070820 0x7ff7101609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.891+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 >> v1:172.21.15.146:6789/0 conn(0x7ff7100d03a0 legacy=0x7ff7100cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7100cf8c0 con 0x7ff7100cdca0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7ff7080019f0 con 0x7ff7100cdca0 2024-04-25T19:08:26.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 >> v1:172.21.15.123:6789/0 conn(0x7ff7100cdca0 legacy=0x7ff71006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f8003a00 0x7ff6f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff6f8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f8005350 0x7ff6f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7100cff10 con 0x7ff6f8004800 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f800c250 con 0x7ff6f8003a00 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f800c450 con 0x7ff6f8005350 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715000640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff6f8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715a00640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f8003a00 0x7ff6f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff716400640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f8005350 0x7ff6f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715000640 1 -- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f8005350 msgr2=0x7ff6f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715000640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f8005350 0x7ff6f800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715000640 1 -- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f8003a00 msgr2=0x7ff6f8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:26.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.892+0000 7ff715000640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f8003a00 0x7ff6f8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.893+0000 7ff715000640 1 -- 172.21.15.123:0/1743199493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f8003520 con 0x7ff6f8004800 2024-04-25T19:08:26.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.893+0000 7ff716400640 1 -- 172.21.15.123:0/1743199493 reap_dead start 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.893+0000 7ff715000640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff6f8004c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00ba30 tx=0x7ff6f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb895bb0dfa3d7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.894+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc030070 con 0x7ff6f8004800 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.894+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6fc02ade0 con 0x7ff6f8004800 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.894+0000 7ff70fe00640 1 -- 172.21.15.123:0/1743199493 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00b600 con 0x7ff6f8004800 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.894+0000 7ff718400640 1 -- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 msgr2=0x7ff6f8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.894+0000 7ff718400640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff6f8004c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00ba30 tx=0x7ff6f8005830 comp rx=0 tx=0).stop 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.895+0000 7ff718400640 1 -- 172.21.15.123:0/1743199493 shutdown_connections 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.895+0000 7ff718400640 1 --2- 172.21.15.123:0/1743199493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff6f8004c10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.895+0000 7ff718400640 1 -- 172.21.15.123:0/1743199493 >> 172.21.15.123:0/1743199493 conn(0x7ff7100ce390 msgr2=0x7ff7100ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.895+0000 7ff718400640 1 -- 172.21.15.123:0/1743199493 shutdown_connections 2024-04-25T19:08:26.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.895+0000 7ff718400640 1 -- 172.21.15.123:0/1743199493 wait complete. 2024-04-25T19:08:26.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.897+0000 7ff718400640 1 Processor -- start 2024-04-25T19:08:26.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.897+0000 7ff718400640 1 -- start start 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7100b7230 0x7ff7101463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7100caa80 0x7ff7100cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7100cbf00 con 0x7ff6f8004800 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7100cbd80 con 0x7ff7100b7230 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.898+0000 7ff718400640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7100cc080 con 0x7ff7100caa80 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57800/0 (socket says 172.21.15.123:57800) 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 -- 172.21.15.123:0/481903356 learned_addr learned my addr 172.21.15.123:0/481903356 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 -- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7100caa80 msgr2=0x7ff7100cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7100caa80 0x7ff7100cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 -- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7100b7230 msgr2=0x7ff7101463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7100b7230 0x7ff7101463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 -- 172.21.15.123:0/481903356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff710161180 con 0x7ff6f8004800 2024-04-25T19:08:26.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff715a00640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e440 tx=0x7ff70400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.899+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704019070 con 0x7ff6f8004800 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.900+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040052e0 con 0x7ff6f8004800 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.900+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704005490 con 0x7ff6f8004800 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.900+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7100cb3d0 con 0x7ff6f8004800 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.900+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71015f620 con 0x7ff6f8004800 2024-04-25T19:08:26.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.901+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff710076a40 con 0x7ff6f8004800 2024-04-25T19:08:26.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.902+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff70401c070 con 0x7ff6f8004800 2024-04-25T19:08:26.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.903+0000 7ff70d600640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 0x7ff6f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.903+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6303+0+0 (secure 0 0 0) 0x7ff7040948c0 con 0x7ff6f8004800 2024-04-25T19:08:26.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.903+0000 7ff715000640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 0x7ff6f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.904+0000 7ff715000640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 0x7ff6f00769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00ba00 tx=0x7ff6fc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:26.908+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7100cb3d0 con 0x7ff6f8004800 2024-04-25T19:08:27.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.130+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff710069c60 con 0x7ff6f8004800 2024-04-25T19:08:27.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.606+0000 7ff70d600640 1 -- 172.21.15.123:0/481903356 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7ff704065e20 con 0x7ff6f8004800 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 msgr2=0x7ff6f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 0x7ff6f00769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00ba00 tx=0x7ff6fc02c000 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 msgr2=0x7ff7100b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e440 tx=0x7ff70400e900 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 shutdown_connections 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff6f00744b0 0x7ff6f00769c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7100caa80 0x7ff7100cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7100b7230 0x7ff7101463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 --2- 172.21.15.123:0/481903356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f8004800 0x7ff7100b44f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 >> 172.21.15.123:0/481903356 conn(0x7ff7100ce390 msgr2=0x7ff71015e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:27.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 shutdown_connections 2024-04-25T19:08:27.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.611+0000 7ff718400640 1 -- 172.21.15.123:0/481903356 wait complete. 2024-04-25T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T19:08:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.910+0000 7f4a8de00640 1 Processor -- start 2024-04-25T19:08:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.910+0000 7f4a8de00640 1 -- start start 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.912+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a8806d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.912+0000 7f4a8de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a8806dca0 0x7f4a8815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.912+0000 7f4a8de00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a8815eab0 0x7f4a88154a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a8de00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8815f460 con 0x7f4a8806dca0 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a8de00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8815eec0 con 0x7f4a8815eab0 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a8de00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8815f180 con 0x7f4a880b53a0 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a8806dca0 0x7f4a8815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a8806dca0 0x7f4a8815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57814/0 (socket says 172.21.15.123:57814) 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 -- 172.21.15.123:0/2516438279 learned_addr learned my addr 172.21.15.123:0/2516438279 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 -- 172.21.15.123:0/2516438279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 msgr2=0x7f4a8806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 --2- 172.21.15.123:0/2516438279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a8806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 -- 172.21.15.123:0/2516438279 >> v2:172.21.15.146:3300/0 conn(0x7f4a8815eab0 msgr2=0x7f4a88154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 --2- 172.21.15.123:0/2516438279 >> v2:172.21.15.146:3300/0 conn(0x7f4a8815eab0 0x7f4a88154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.913+0000 7f4a86000640 1 -- 172.21.15.123:0/2516438279 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880b48c0 con 0x7f4a8806dca0 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.914+0000 7f4a86000640 1 --2- 172.21.15.123:0/2516438279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a8815e570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4a880d1770 tx=0x7f4a7002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838aa9407ab3409e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.915+0000 7f4a85600640 1 -- 172.21.15.123:0/2516438279 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70037070 con 0x7f4a8806dca0 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.915+0000 7f4a85600640 1 -- 172.21.15.123:0/2516438279 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70032440 con 0x7f4a8806dca0 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.915+0000 7f4a85600640 1 -- 172.21.15.123:0/2516438279 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70032810 con 0x7f4a8806dca0 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 -- 172.21.15.123:0/2516438279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 msgr2=0x7f4a8815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2516438279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a8815e570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4a880d1770 tx=0x7f4a7002a140 comp rx=0 tx=0).stop 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 -- 172.21.15.123:0/2516438279 shutdown_connections 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2516438279 >> v2:172.21.15.146:3300/0 conn(0x7f4a8815eab0 0x7f4a88154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2516438279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a8815e570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2516438279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a8806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 -- 172.21.15.123:0/2516438279 >> 172.21.15.123:0/2516438279 conn(0x7f4a880d1f80 msgr2=0x7f4a880732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 -- 172.21.15.123:0/2516438279 shutdown_connections 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.916+0000 7f4a8de00640 1 -- 172.21.15.123:0/2516438279 wait complete. 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.919+0000 7f4a8de00640 1 Processor -- start 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.919+0000 7f4a8de00640 1 -- start start 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a88149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a8815eab0 0x7f4a8815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88154f60 con 0x7f4a8806dca0 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880b4e80 con 0x7f4a8815eab0 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.921+0000 7f4a8de00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a881550e0 con 0x7f4a880b53a0 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57830/0 (socket says 172.21.15.123:57830) 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 -- 172.21.15.123:0/2263615242 learned_addr learned my addr 172.21.15.123:0/2263615242 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 -- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 msgr2=0x7f4a88149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a88149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 -- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a8815eab0 msgr2=0x7f4a8815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a8815eab0 0x7f4a8815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 -- 172.21.15.123:0/2263615242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a881556d0 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a86a00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000ba50 tx=0x7f4a8000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.922+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80019070 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.923+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8000cba0 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.923+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80014440 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.923+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a88159cb0 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.923+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8815a270 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.924+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a88076a40 con 0x7f4a8806dca0 2024-04-25T19:08:27.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.925+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a8001c070 con 0x7f4a8806dca0 2024-04-25T19:08:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.926+0000 7f4a7f400640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 0x7f4a640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.926+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6303+0+0 (secure 0 0 0) 0x7f4a800952c0 con 0x7f4a8806dca0 2024-04-25T19:08:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.926+0000 7f4a86000640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 0x7f4a640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.927+0000 7f4a86000640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 0x7f4a640769f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a70034000 tx=0x7f4a7002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:27.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:27.932+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a8005f760 con 0x7f4a8806dca0 2024-04-25T19:08:28.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.133+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a8815a4b0 con 0x7f4a8806dca0 2024-04-25T19:08:28.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.885+0000 7f4a7f400640 1 -- 172.21.15.123:0/2263615242 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7f4a8005f060 con 0x7f4a8806dca0 2024-04-25T19:08:28.887 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-04-25T19:08:28.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 msgr2=0x7f4a640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:28.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 0x7f4a640769f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a70034000 tx=0x7f4a7002c000 comp rx=0 tx=0).stop 2024-04-25T19:08:28.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 msgr2=0x7f4a88157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:28.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000ba50 tx=0x7f4a8000bf10 comp rx=0 tx=0).stop 2024-04-25T19:08:28.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 shutdown_connections 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f4a640744e0 0x7f4a640769f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a8815eab0 0x7f4a8815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a880b53a0 0x7f4a88149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 --2- 172.21.15.123:0/2263615242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a8806dca0 0x7f4a88157fe0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.889+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 >> 172.21.15.123:0/2263615242 conn(0x7f4a880d1f80 msgr2=0x7f4a880732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.890+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 shutdown_connections 2024-04-25T19:08:28.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:28.890+0000 7f4a8de00640 1 -- 172.21.15.123:0/2263615242 wait complete. 2024-04-25T19:08:28.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T19:08:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.194+0000 7f80e3200640 1 Processor -- start 2024-04-25T19:08:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.194+0000 7f80e3200640 1 -- start start 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80dc070ac0 0x7f80dc15d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e3200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80dc06d580 0x7f80dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80dc06ded0 con 0x7f80dc0b4b00 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f80dc0b6550 con 0x7f80dc070ac0 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e3200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f80dc0b6810 con 0x7f80dc06d580 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e1e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80dc070ac0 0x7f80dc15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e1400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f80dc0b4b00 0x7f80dc0cf2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55514/0 (socket says 172.21.15.123:55514) 2024-04-25T19:08:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.197+0000 7f80e1400640 1 -- 172.21.15.123:0/1300031699 learned_addr learned my addr 172.21.15.123:0/1300031699 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e2800640 1 --2- 172.21.15.123:0/1300031699 >> v2:172.21.15.146:3300/0 conn(0x7f80dc06d580 0x7f80dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e1e00640 1 -- 172.21.15.123:0/1300031699 >> v2:172.21.15.146:3300/0 conn(0x7f80dc06d580 msgr2=0x7f80dc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e1e00640 1 --2- 172.21.15.123:0/1300031699 >> v2:172.21.15.146:3300/0 conn(0x7f80dc06d580 0x7f80dc06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e1e00640 1 -- 172.21.15.123:0/1300031699 >> v1:172.21.15.123:6789/0 conn(0x7f80dc0b4b00 legacy=0x7f80dc0cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e1e00640 1 -- 172.21.15.123:0/1300031699 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc15e320 con 0x7f80dc070ac0 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.198+0000 7f80e0a00640 1 -- 172.21.15.123:0/1300031699 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053960704 0 0) 0x7f80c8001110 con 0x7f80dc0b4b00 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.199+0000 7f80e1e00640 1 --2- 172.21.15.123:0/1300031699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc070ac0 0x7f80dc15d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f80dc072480 tx=0x7f80d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75177d20de0d81d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.199+0000 7f80e0a00640 1 -- 172.21.15.123:0/1300031699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0037070 con 0x7f80dc070ac0 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e0a00640 1 -- 172.21.15.123:0/1300031699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d0032440 con 0x7f80dc070ac0 2024-04-25T19:08:29.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e0a00640 1 -- 172.21.15.123:0/1300031699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0032810 con 0x7f80dc070ac0 2024-04-25T19:08:29.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 -- 172.21.15.123:0/1300031699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc070ac0 msgr2=0x7f80dc15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 --2- 172.21.15.123:0/1300031699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc070ac0 0x7f80dc15d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f80dc072480 tx=0x7f80d002a1a0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 -- 172.21.15.123:0/1300031699 shutdown_connections 2024-04-25T19:08:29.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 --2- 172.21.15.123:0/1300031699 >> v2:172.21.15.146:3300/0 conn(0x7f80dc06d580 0x7f80dc06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 --2- 172.21.15.123:0/1300031699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc070ac0 0x7f80dc15d330 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 -- 172.21.15.123:0/1300031699 >> 172.21.15.123:0/1300031699 conn(0x7f80dc072fd0 msgr2=0x7f80dc0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.200+0000 7f80e3200640 1 -- 172.21.15.123:0/1300031699 shutdown_connections 2024-04-25T19:08:29.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.201+0000 7f80e3200640 1 -- 172.21.15.123:0/1300031699 wait complete. 2024-04-25T19:08:29.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.203+0000 7f80e3200640 1 Processor -- start 2024-04-25T19:08:29.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.203+0000 7f80e3200640 1 -- start start 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80dc06d580 0x7f80dc1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80dc070ac0 0x7f80dc147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc15e040 con 0x7f80dc0b4b00 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc15dd60 con 0x7f80dc06d580 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.204+0000 7f80e3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f80dc0b4f10 con 0x7f80dc070ac0 2024-04-25T19:08:29.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57834/0 (socket says 172.21.15.123:57834) 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 -- 172.21.15.123:0/4217190312 learned_addr learned my addr 172.21.15.123:0/4217190312 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e1e00640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80dc06d580 0x7f80dc1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e1400640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80dc070ac0 0x7f80dc147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 -- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80dc070ac0 msgr2=0x7f80dc147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80dc070ac0 0x7f80dc147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 -- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80dc06d580 msgr2=0x7f80dc1476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80dc06d580 0x7f80dc1476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.205+0000 7f80e2800640 1 -- 172.21.15.123:0/4217190312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc149660 con 0x7f80dc0b4b00 2024-04-25T19:08:29.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80e2800640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f80d8007d40 tx=0x7f80d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8014420 con 0x7f80dc0b4b00 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8019070 con 0x7f80dc0b4b00 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8014820 con 0x7f80dc0b4b00 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc149940 con 0x7f80dc0b4b00 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.206+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc14fd50 con 0x7f80dc0b4b00 2024-04-25T19:08:29.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.207+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80dc0b6190 con 0x7f80dc0b4b00 2024-04-25T19:08:29.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.208+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80d80149c0 con 0x7f80dc0b4b00 2024-04-25T19:08:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.210+0000 7f80d6000640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 0x7f80bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.210+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5865+0+0 (secure 0 0 0) 0x7f80d8095b70 con 0x7f80dc0b4b00 2024-04-25T19:08:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.210+0000 7f80e1e00640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 0x7f80bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.211+0000 7f80e1e00640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 0x7f80bc0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80d0034000 tx=0x7f80d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.215+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80dc0b6190 con 0x7f80dc0b4b00 2024-04-25T19:08:29.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.416+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f80dc050740 con 0x7f80dc0b4b00 2024-04-25T19:08:29.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.924+0000 7f80d6000640 1 -- 172.21.15.123:0/4217190312 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7f80d80601c0 con 0x7f80dc0b4b00 2024-04-25T19:08:29.927 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-04-25T19:08:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 msgr2=0x7f80bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 0x7f80bc0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80d0034000 tx=0x7f80d002e000 comp rx=0 tx=0).stop 2024-04-25T19:08:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 msgr2=0x7f80dc14b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f80d8007d40 tx=0x7f80d800e4b0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 shutdown_connections 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f80bc0744b0 0x7f80bc0769c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80dc0b4b00 0x7f80dc14b350 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80dc070ac0 0x7f80dc147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 --2- 172.21.15.123:0/4217190312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80dc06d580 0x7f80dc1476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.928+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 >> 172.21.15.123:0/4217190312 conn(0x7f80dc072fd0 msgr2=0x7f80dc1558f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.929+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 shutdown_connections 2024-04-25T19:08:29.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:29.929+0000 7f80e3200640 1 -- 172.21.15.123:0/4217190312 wait complete. 2024-04-25T19:08:29.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T19:08:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.220+0000 7f7015600640 1 Processor -- start 2024-04-25T19:08:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.221+0000 7f7015600640 1 -- start start 2024-04-25T19:08:30.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70100b4640 0x7f701015d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70100d0f70 0x7f7010160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7010160070 con 0x7f70100b4640 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f701006d750 con 0x7f70100d0f70 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f7015600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70100d1380 con 0x7f701006ce00 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f700fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70100b4640 0x7f701015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f700fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70100b4640 0x7f701015d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57850/0 (socket says 172.21.15.123:57850) 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.223+0000 7f700fe00640 1 -- 172.21.15.123:0/3252167629 learned_addr learned my addr 172.21.15.123:0/3252167629 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f7014c00640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f700fe00640 1 -- 172.21.15.123:0/3252167629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 msgr2=0x7f701006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f700fe00640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701006d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f700fe00640 1 -- 172.21.15.123:0/3252167629 >> v2:172.21.15.146:3300/0 conn(0x7f70100d0f70 msgr2=0x7f7010160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f700fe00640 1 --2- 172.21.15.123:0/3252167629 >> v2:172.21.15.146:3300/0 conn(0x7f70100d0f70 0x7f7010160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.224+0000 7f700fe00640 1 -- 172.21.15.123:0/3252167629 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70100b66f0 con 0x7f70100b4640 2024-04-25T19:08:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.225+0000 7f700fe00640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f701015d500 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7010074460 tx=0x7f700002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e437e0538ff54ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.225+0000 7f700ea00640 1 -- 172.21.15.123:0/3252167629 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000037070 con 0x7f70100b4640 2024-04-25T19:08:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.225+0000 7f700ea00640 1 -- 172.21.15.123:0/3252167629 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700002ae20 con 0x7f70100b4640 2024-04-25T19:08:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.225+0000 7f700ea00640 1 -- 172.21.15.123:0/3252167629 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000032660 con 0x7f70100b4640 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 -- 172.21.15.123:0/3252167629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 msgr2=0x7f701015d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f701015d500 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7010074460 tx=0x7f700002a110 comp rx=0 tx=0).stop 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 -- 172.21.15.123:0/3252167629 shutdown_connections 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 --2- 172.21.15.123:0/3252167629 >> v2:172.21.15.146:3300/0 conn(0x7f70100d0f70 0x7f7010160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 --2- 172.21.15.123:0/3252167629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f701015d500 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 -- 172.21.15.123:0/3252167629 >> 172.21.15.123:0/3252167629 conn(0x7f70100d19d0 msgr2=0x7f70100d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 -- 172.21.15.123:0/3252167629 shutdown_connections 2024-04-25T19:08:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.226+0000 7f7015600640 1 -- 172.21.15.123:0/3252167629 wait complete. 2024-04-25T19:08:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.228+0000 7f7015600640 1 Processor -- start 2024-04-25T19:08:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.229+0000 7f7015600640 1 -- start start 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70100d0f70 0x7f701015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f701006c570 con 0x7f70100b4640 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f70101602c0 con 0x7f70100d0f70 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f7015600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f701006c6f0 con 0x7f701006ce00 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.230+0000 7f700f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57854/0 (socket says 172.21.15.123:57854) 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 -- 172.21.15.123:0/670242349 learned_addr learned my addr 172.21.15.123:0/670242349 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700fe00640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 -- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 msgr2=0x7f701014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 -- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70100d0f70 msgr2=0x7f701015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70100d0f70 0x7f701015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 -- 172.21.15.123:0/670242349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7010158610 con 0x7f70100b4640 2024-04-25T19:08:30.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f700f400640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b440 tx=0x7f6ff8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.231+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8015020 con 0x7f70100b4640 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.232+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011830 con 0x7f70100b4640 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.232+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011c20 con 0x7f70100b4640 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.232+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701014e6e0 con 0x7f70100b4640 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.232+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7010158050 con 0x7f70100b4640 2024-04-25T19:08:30.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.233+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7010050740 con 0x7f70100b4640 2024-04-25T19:08:30.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.234+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ff8018020 con 0x7f70100b4640 2024-04-25T19:08:30.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.235+0000 7f7007e00640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 0x7f6fec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.235+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6ff8095080 con 0x7f70100b4640 2024-04-25T19:08:30.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.236+0000 7f700fe00640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 0x7f6fec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.236+0000 7f700fe00640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 0x7f6fec0769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7000034000 tx=0x7f700002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.240+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ff801f020 con 0x7f70100b4640 2024-04-25T19:08:30.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.437+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7010157ac0 con 0x7f70100b4640 2024-04-25T19:08:30.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.445+0000 7f7007e00640 1 -- 172.21.15.123:0/670242349 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f6ff805f290 con 0x7f70100b4640 2024-04-25T19:08:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 msgr2=0x7f6fec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 0x7f6fec0769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7000034000 tx=0x7f700002e000 comp rx=0 tx=0).stop 2024-04-25T19:08:30.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 msgr2=0x7f70101570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b440 tx=0x7f6ff8009f90 comp rx=0 tx=0).stop 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 shutdown_connections 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f6fec0744b0 0x7f6fec0769c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70100d0f70 0x7f701015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70100b4640 0x7f70101570b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 --2- 172.21.15.123:0/670242349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701006ce00 0x7f701014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 >> 172.21.15.123:0/670242349 conn(0x7f70100d19d0 msgr2=0x7f70100d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 shutdown_connections 2024-04-25T19:08:30.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.447+0000 7f7015600640 1 -- 172.21.15.123:0/670242349 wait complete. 2024-04-25T19:08:30.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-25T19:08:30.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.749+0000 7fdeb3e00640 1 Processor -- start 2024-04-25T19:08:30.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.749+0000 7fdeb3e00640 1 -- start start 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdeb406dc90 0x7fdeb4142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdeb4142a70 0x7fdeb40b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb40b5c70 con 0x7fdeb413fbf0 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40b5f50 con 0x7fdeb4142a70 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40b6230 con 0x7fdeb406dc90 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdeb4142a70 0x7fdeb40b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdeb4142a70 0x7fdeb40b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57866/0 (socket says 172.21.15.123:57866) 2024-04-25T19:08:30.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.751+0000 7fdeb3400640 1 -- 172.21.15.123:0/793864428 learned_addr learned my addr 172.21.15.123:0/793864428 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:30.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.752+0000 7fdeb3400640 1 -- 172.21.15.123:0/793864428 >> v2:172.21.15.146:3300/0 conn(0x7fdeb406dc90 msgr2=0x7fdeb4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.752+0000 7fdeb3400640 1 --2- 172.21.15.123:0/793864428 >> v2:172.21.15.146:3300/0 conn(0x7fdeb406dc90 0x7fdeb4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.752+0000 7fdeb3400640 1 -- 172.21.15.123:0/793864428 >> v1:172.21.15.146:6789/0 conn(0x7fdeb413fbf0 legacy=0x7fdeb406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.752+0000 7fdeb3400640 1 -- 172.21.15.123:0/793864428 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb40b6510 con 0x7fdeb4142a70 2024-04-25T19:08:30.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.753+0000 7fdeb3400640 1 --2- 172.21.15.123:0/793864428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb40b4640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4070770 tx=0x7fdeac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61855a556dfde70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.753+0000 7fdeb1600640 1 -- 172.21.15.123:0/793864428 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac037070 con 0x7fdeb4142a70 2024-04-25T19:08:30.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.753+0000 7fdeb1600640 1 -- 172.21.15.123:0/793864428 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac032440 con 0x7fdeb4142a70 2024-04-25T19:08:30.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.753+0000 7fdeb1600640 1 -- 172.21.15.123:0/793864428 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac032810 con 0x7fdeb4142a70 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 -- 172.21.15.123:0/793864428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 msgr2=0x7fdeb40b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/793864428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb40b4640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4070770 tx=0x7fdeac02a230 comp rx=0 tx=0).stop 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 -- 172.21.15.123:0/793864428 shutdown_connections 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/793864428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb40b4640 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/793864428 >> v2:172.21.15.146:3300/0 conn(0x7fdeb406dc90 0x7fdeb4142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 -- 172.21.15.123:0/793864428 >> 172.21.15.123:0/793864428 conn(0x7fdeb4073200 msgr2=0x7fdeb4073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 -- 172.21.15.123:0/793864428 shutdown_connections 2024-04-25T19:08:30.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.754+0000 7fdeb3e00640 1 -- 172.21.15.123:0/793864428 wait complete. 2024-04-25T19:08:30.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.756+0000 7fdeb3e00640 1 Processor -- start 2024-04-25T19:08:30.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.757+0000 7fdeb3e00640 1 -- start start 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb413fbf0 0x7fdeb41336d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdeb4160e50 0x7fdeb414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4143000 con 0x7fdeb4142a70 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4142e80 con 0x7fdeb413fbf0 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb3e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb40b4b80 con 0x7fdeb4160e50 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57876/0 (socket says 172.21.15.123:57876) 2024-04-25T19:08:30.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.758+0000 7fdeb2000640 1 -- 172.21.15.123:0/257889492 learned_addr learned my addr 172.21.15.123:0/257889492 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 -- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdeb4160e50 msgr2=0x7fdeb414d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb3400640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdeb4160e50 0x7fdeb414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdeb4160e50 0x7fdeb414d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2a00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb413fbf0 0x7fdeb41336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 -- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb413fbf0 msgr2=0x7fdeb41336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb413fbf0 0x7fdeb41336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 -- 172.21.15.123:0/257889492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb4140fb0 con 0x7fdeb4142a70 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb2000640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b410 tx=0x7fde9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c015020 con 0x7fdeb4142a70 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9c011850 con 0x7fdeb4142a70 2024-04-25T19:08:30.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.759+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb4141290 con 0x7fdeb4142a70 2024-04-25T19:08:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.760+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c011c40 con 0x7fdeb4142a70 2024-04-25T19:08:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.760+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb4141850 con 0x7fdeb4142a70 2024-04-25T19:08:30.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.761+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb4050740 con 0x7fdeb4142a70 2024-04-25T19:08:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.762+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde9c018020 con 0x7fdeb4142a70 2024-04-25T19:08:30.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.763+0000 7fdeaaa00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 0x7fde900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.763+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fde9c095120 con 0x7fdeb4142a70 2024-04-25T19:08:30.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.763+0000 7fdeb2a00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 0x7fde900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.764+0000 7fdeb2a00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 0x7fde900769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007dc0 tx=0x7fdea40072f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:30.768+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde9c01f020 con 0x7fdeb4142a70 2024-04-25T19:08:31.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.039+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fdeb40cfb70 con 0x7fdeb4142a70 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.942+0000 7fdeaaa00640 1 -- 172.21.15.123:0/257889492 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v262) ==== 120+0+0 (secure 0 0 0) 0x7fde9c05f980 con 0x7fdeb4142a70 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.946+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 msgr2=0x7fde900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.946+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 0x7fde900769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007dc0 tx=0x7fdea40072f0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.946+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 msgr2=0x7fdeb4160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.946+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b410 tx=0x7fde9c00b8d0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 shutdown_connections 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fde900744b0 0x7fde900769c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdeb4160e50 0x7fdeb414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb4142a70 0x7fdeb4160910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 --2- 172.21.15.123:0/257889492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb413fbf0 0x7fdeb41336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 >> 172.21.15.123:0/257889492 conn(0x7fdeb4073200 msgr2=0x7fdeb40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 shutdown_connections 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:31.947+0000 7fdeb3e00640 1 -- 172.21.15.123:0/257889492 wait complete. 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner: 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner:Ran 1 test in 286.359s 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner: 2024-04-25T19:08:31.976 INFO:tasks.cephfs_test_runner:OK 2024-04-25T19:08:31.977 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T19:08:31.986 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T19:08:32.044 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T19:08:32.044 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:08:32.044 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:32.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T19:08:32.101 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-25T19:08:32.101 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:32.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T19:08:32.154 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:08:32.154 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:32.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T19:08:32.211 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T19:08:32.211 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:08:32.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T19:08:32.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.265 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:08:32.322 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:08:32.322 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T19:08:32.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:08:32.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T19:08:32.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T19:08:32.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.595 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-25T19:08:32.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-25T19:08:32.758 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-04-25T19:08:32.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:08:32.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.762 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:08:32.819 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T19:08:32.819 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T19:08:32.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.820 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T19:08:32.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.905 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T19:08:32.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:32.978 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T19:08:33.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:33.079 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T19:08:33.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:08:33.120 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T19:08:33.242 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T19:08:33.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:08:33.246 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T19:08:33.256 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T19:08:33.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:08:33.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.542+0000 7ff107e00640 1 Processor -- start 2024-04-25T19:08:33.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.543+0000 7ff107e00640 1 -- start start 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff10806dc90 0x7ff1080b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1081456e0 con 0x7ff1080b0af0 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1081459c0 con 0x7ff1081450d0 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff1081462f0 con 0x7ff10806dc90 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff1080b0af0 0x7ff108146ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55520/0 (socket says 172.21.15.123:55520) 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.545+0000 7ff107400640 1 -- 172.21.15.123:0/2725132840 learned_addr learned my addr 172.21.15.123:0/2725132840 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:33.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff0ffe00640 1 --2- 172.21.15.123:0/2725132840 >> v2:172.21.15.123:3300/0 conn(0x7ff10806dc90 0x7ff1080b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:33.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff106000640 1 -- 172.21.15.123:0/2725132840 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896193093 0 0) 0x7ff1081456e0 con 0x7ff1080b0af0 2024-04-25T19:08:33.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff106000640 1 -- 172.21.15.123:0/2725132840 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ec003520 con 0x7ff1080b0af0 2024-04-25T19:08:33.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff0ffe00640 1 -- 172.21.15.123:0/2725132840 >> v1:172.21.15.146:6789/0 conn(0x7ff1081450d0 legacy=0x7ff10806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff0ffe00640 1 -- 172.21.15.123:0/2725132840 >> v1:172.21.15.123:6789/0 conn(0x7ff1080b0af0 legacy=0x7ff108146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.546+0000 7ff0ffe00640 1 -- 172.21.15.123:0/2725132840 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff108145ca0 con 0x7ff10806dc90 2024-04-25T19:08:33.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.547+0000 7ff0ffe00640 1 --2- 172.21.15.123:0/2725132840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff10806dc90 0x7ff1080b05b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff108146470 tx=0x7ff0f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e2646f56d82afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:33.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.548+0000 7ff106000640 1 -- 172.21.15.123:0/2725132840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037070 con 0x7ff10806dc90 2024-04-25T19:08:33.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.548+0000 7ff106000640 1 -- 172.21.15.123:0/2725132840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f0032440 con 0x7ff10806dc90 2024-04-25T19:08:33.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.548+0000 7ff106000640 1 -- 172.21.15.123:0/2725132840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0032810 con 0x7ff10806dc90 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.548+0000 7ff107e00640 1 -- 172.21.15.123:0/2725132840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff10806dc90 msgr2=0x7ff1080b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.548+0000 7ff107e00640 1 --2- 172.21.15.123:0/2725132840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff10806dc90 0x7ff1080b05b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff108146470 tx=0x7ff0f002a210 comp rx=0 tx=0).stop 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.549+0000 7ff107e00640 1 -- 172.21.15.123:0/2725132840 shutdown_connections 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.549+0000 7ff107e00640 1 --2- 172.21.15.123:0/2725132840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff10806dc90 0x7ff1080b05b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.549+0000 7ff107e00640 1 -- 172.21.15.123:0/2725132840 >> 172.21.15.123:0/2725132840 conn(0x7ff108073200 msgr2=0x7ff108073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.549+0000 7ff107e00640 1 -- 172.21.15.123:0/2725132840 shutdown_connections 2024-04-25T19:08:33.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.549+0000 7ff107e00640 1 -- 172.21.15.123:0/2725132840 wait complete. 2024-04-25T19:08:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.551+0000 7ff107e00640 1 Processor -- start 2024-04-25T19:08:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.551+0000 7ff107e00640 1 -- start start 2024-04-25T19:08:33.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 0x7ff108141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:33.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 0x7ff108139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:33.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080b1080 con 0x7ff1081611a0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080b0f00 con 0x7ff1081450d0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1081471e0 con 0x7ff1080b0af0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57902/0 (socket says 172.21.15.123:57902) 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107400640 1 -- 172.21.15.123:0/699327987 learned_addr learned my addr 172.21.15.123:0/699327987 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff0ffe00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 0x7ff108139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff107400640 1 -- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 msgr2=0x7ff108141980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.553+0000 7ff106a00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 0x7ff108141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107400640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 0x7ff108141980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107400640 1 -- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 msgr2=0x7ff108139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107400640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 0x7ff108139460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107400640 1 -- 172.21.15.123:0/699327987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1081e9a40 con 0x7ff1081611a0 2024-04-25T19:08:33.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff0ffe00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 0x7ff108139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff106a00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 0x7ff108141980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107400640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff10000caa0 tx=0x7ff10000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100017070 con 0x7ff1081611a0 2024-04-25T19:08:33.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10000ac00 con 0x7ff1081611a0 2024-04-25T19:08:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100012400 con 0x7ff1081611a0 2024-04-25T19:08:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1081e9ce0 con 0x7ff1081611a0 2024-04-25T19:08:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.554+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1080cf000 con 0x7ff1081611a0 2024-04-25T19:08:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.556+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1081ea200 con 0x7ff1081611a0 2024-04-25T19:08:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.557+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff10001a070 con 0x7ff1081611a0 2024-04-25T19:08:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.558+0000 7ff0fea00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 0x7ff0e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:33.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.558+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7ff100094cc0 con 0x7ff1081611a0 2024-04-25T19:08:33.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.558+0000 7ff106a00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 0x7ff0e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:33.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.559+0000 7ff106a00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 0x7ff0e40769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80015e0 tx=0x7ff0f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:33.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.562+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1081ea200 con 0x7ff1081611a0 2024-04-25T19:08:33.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.760+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1080cd8a0 con 0x7ff0e40744b0 2024-04-25T19:08:33.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.762+0000 7ff0fea00640 1 -- 172.21.15.123:0/699327987 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7ff1080cd8a0 con 0x7ff0e40744b0 2024-04-25T19:08:33.764 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:33.764 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 msgr2=0x7ff0e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 0x7ff0e40769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80015e0 tx=0x7ff0f8009040 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 msgr2=0x7ff108160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff10000caa0 tx=0x7ff10000cf60 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 shutdown_connections 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.765+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7ff0e40744b0 0x7ff0e40769c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1081611a0 0x7ff108160470 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1081450d0 0x7ff108139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 --2- 172.21.15.123:0/699327987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1080b0af0 0x7ff108141980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 >> 172.21.15.123:0/699327987 conn(0x7ff108073200 msgr2=0x7ff108144250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 shutdown_connections 2024-04-25T19:08:33.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:33.766+0000 7ff107e00640 1 -- 172.21.15.123:0/699327987 wait complete. 2024-04-25T19:08:33.795 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-25T19:08:33.087064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225604,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749116,"statfs":{"total":773094113280,"available":772863094784,"internally_reserved":0,"allocated":5382144,"data_stored":2993480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617455,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-121,"num_write_kb":-663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.482392"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T19:08:29.974555+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:08:29.974555+0000","last_peered":"2024-04-25T19:08:29.974555+0000","last_clean":"2024-04-25T19:08:29.974555+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:08:29.974555+0000","last_undegraded":"2024-04-25T19:08:29.974555+0000","last_fullsized":"2024-04-25T19:08:29.974555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:08:33.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:08:34.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.054+0000 7f59d2600640 1 Processor -- start 2024-04-25T19:08:34.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.054+0000 7f59d2600640 1 -- start start 2024-04-25T19:08:34.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59cc06d580 0x7f59cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59cc15e2f0 0x7f59cc15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f59cc06ded0 con 0x7f59cc15e2f0 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f59cc15f070 con 0x7f59cc06d580 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d2600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc06f340 con 0x7f59cc06fe20 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d1c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59cc15e2f0 0x7f59cc15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d1c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59cc15e2f0 0x7f59cc15e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57910/0 (socket says 172.21.15.123:57910) 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.057+0000 7f59d1c00640 1 -- 172.21.15.123:0/3589693191 learned_addr learned my addr 172.21.15.123:0/3589693191 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1c00640 1 -- 172.21.15.123:0/3589693191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 msgr2=0x7f59cc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1200640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1c00640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1c00640 1 -- 172.21.15.123:0/3589693191 >> v2:172.21.15.146:3300/0 conn(0x7f59cc06d580 msgr2=0x7f59cc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1c00640 1 --2- 172.21.15.123:0/3589693191 >> v2:172.21.15.146:3300/0 conn(0x7f59cc06d580 0x7f59cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1c00640 1 -- 172.21.15.123:0/3589693191 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc06f620 con 0x7f59cc15e2f0 2024-04-25T19:08:34.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.058+0000 7f59d1200640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:34.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.059+0000 7f59d1c00640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc15e2f0 0x7f59cc15e700 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59cc074430 tx=0x7f59c402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b2f9fe73ad4920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.059+0000 7f59cb400640 1 -- 172.21.15.123:0/3589693191 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4036070 con 0x7f59cc15e2f0 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.059+0000 7f59cb400640 1 -- 172.21.15.123:0/3589693191 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4031600 con 0x7f59cc15e2f0 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.059+0000 7f59cb400640 1 -- 172.21.15.123:0/3589693191 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c40319d0 con 0x7f59cc15e2f0 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 -- 172.21.15.123:0/3589693191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc15e2f0 msgr2=0x7f59cc15e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc15e2f0 0x7f59cc15e700 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59cc074430 tx=0x7f59c402a610 comp rx=0 tx=0).stop 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 -- 172.21.15.123:0/3589693191 shutdown_connections 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc15e2f0 0x7f59cc15e700 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 --2- 172.21.15.123:0/3589693191 >> v2:172.21.15.146:3300/0 conn(0x7f59cc06d580 0x7f59cc06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 --2- 172.21.15.123:0/3589693191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 -- 172.21.15.123:0/3589693191 >> 172.21.15.123:0/3589693191 conn(0x7f59cc073200 msgr2=0x7f59cc073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:34.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 -- 172.21.15.123:0/3589693191 shutdown_connections 2024-04-25T19:08:34.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.060+0000 7f59d2600640 1 -- 172.21.15.123:0/3589693191 wait complete. 2024-04-25T19:08:34.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.063+0000 7f59d2600640 1 Processor -- start 2024-04-25T19:08:34.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.063+0000 7f59d2600640 1 -- start start 2024-04-25T19:08:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 0x7f59cc159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc15ec40 con 0x7f59cc06d580 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc15bbf0 con 0x7f59cc15e2f0 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d2600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc15edc0 con 0x7f59cc06fe20 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.064+0000 7f59d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57912/0 (socket says 172.21.15.123:57912) 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 -- 172.21.15.123:0/2457689086 learned_addr learned my addr 172.21.15.123:0/2457689086 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1c00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 0x7f59cc159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 -- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 msgr2=0x7f59cc0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59cbe00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc0cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 -- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 msgr2=0x7f59cc159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 0x7f59cc159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 -- 172.21.15.123:0/2457689086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc1e7cc0 con 0x7f59cc06d580 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1c00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 0x7f59cc159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.065+0000 7f59d1200640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c7f0 tx=0x7f59bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.066+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc015070 con 0x7f59cc06d580 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.066+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc007820 con 0x7f59cc06d580 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.066+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc007bf0 con 0x7f59cc06d580 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.066+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc1e7fa0 con 0x7f59cc06d580 2024-04-25T19:08:34.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.066+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1e8560 con 0x7f59cc06d580 2024-04-25T19:08:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.067+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc050740 con 0x7f59cc06d580 2024-04-25T19:08:34.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.068+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59bc018070 con 0x7f59cc06d580 2024-04-25T19:08:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.069+0000 7f59c8c00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.069+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f59bc0951b0 con 0x7f59cc06d580 2024-04-25T19:08:34.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.069+0000 7f59cbe00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.070+0000 7f59cbe00640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 0x7f59ac0769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59cc1e95e0 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.074+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59bc05faa0 con 0x7f59cc06d580 2024-04-25T19:08:34.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.270+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59cc070a00 con 0x7f59ac0744e0 2024-04-25T19:08:34.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.272+0000 7f59c8c00640 1 -- 172.21.15.123:0/2457689086 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7f59cc070a00 con 0x7f59ac0744e0 2024-04-25T19:08:34.274 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:34.274 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.274+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 msgr2=0x7f59ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.274+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 0x7f59ac0769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59cc1e95e0 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 msgr2=0x7f59cc1e8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c7f0 tx=0x7f59bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 shutdown_connections 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59cc15e2f0 0x7f59cc159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59cc06fe20 0x7f59cc0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 --2- 172.21.15.123:0/2457689086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59cc06d580 0x7f59cc1e8d60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 >> 172.21.15.123:0/2457689086 conn(0x7f59cc073200 msgr2=0x7f59cc15cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:34.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 shutdown_connections 2024-04-25T19:08:34.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.275+0000 7f59d2600640 1 -- 172.21.15.123:0/2457689086 wait complete. 2024-04-25T19:08:34.304 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-25T19:08:33.087064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225604,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749116,"statfs":{"total":773094113280,"available":772863094784,"internally_reserved":0,"allocated":5382144,"data_stored":2993480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617455,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-121,"num_write_kb":-663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.482392"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T19:08:29.974555+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:08:29.974555+0000","last_peered":"2024-04-25T19:08:29.974555+0000","last_clean":"2024-04-25T19:08:29.974555+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:08:29.974555+0000","last_undegraded":"2024-04-25T19:08:29.974555+0000","last_fullsized":"2024-04-25T19:08:29.974555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:08:34.305 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T19:08:34.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:08:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.569+0000 7f449d000640 1 Processor -- start 2024-04-25T19:08:34.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.569+0000 7f449d000640 1 -- start start 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f449d000640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f449806fe20 0x7f449815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f449d000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449815f070 con 0x7f449806d580 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f449d000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449806f340 con 0x7f449815e2f0 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f449d000640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f449815ee10 con 0x7f449806fe20 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f4496a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f449806d580 0x7f449806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55528/0 (socket says 172.21.15.123:55528) 2024-04-25T19:08:34.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.571+0000 7f4496a00640 1 -- 172.21.15.123:0/1755461951 learned_addr learned my addr 172.21.15.123:0/1755461951 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634817912 0 0) 0x7f449815f070 con 0x7f449806d580 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4497400640 1 --2- 172.21.15.123:0/1755461951 >> v2:172.21.15.146:3300/0 conn(0x7f449806fe20 0x7f449815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f447c003520 con 0x7f449806d580 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2510592863 0 0) 0x7f447c003520 con 0x7f449806d580 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 >> v2:172.21.15.146:3300/0 conn(0x7f449806fe20 msgr2=0x7f449815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 --2- 172.21.15.123:0/1755461951 >> v2:172.21.15.146:3300/0 conn(0x7f449806fe20 0x7f449815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 >> v1:172.21.15.146:6789/0 conn(0x7f449815e2f0 legacy=0x7f449815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449806f620 con 0x7f449806d580 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.572+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f4484001670 con 0x7f449806d580 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 >> v1:172.21.15.123:6789/0 conn(0x7f449806d580 legacy=0x7f449806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c003a00 0x7f447c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 0x7f447c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c0052a0 0x7f447c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f449815ee10 con 0x7f447c004800 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f447c010640 con 0x7f447c003a00 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f447c010840 con 0x7f447c0052a0 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 0x7f447c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4497400640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c003a00 0x7f447c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 -- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c0052a0 msgr2=0x7f447c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4497e00640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c0052a0 0x7f447c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c0052a0 0x7f447c00fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 -- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c003a00 msgr2=0x7f447c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c003a00 0x7f447c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.573+0000 7f4496a00640 1 -- 172.21.15.123:0/1755461951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c003520 con 0x7f447c004800 2024-04-25T19:08:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.574+0000 7f4497e00640 1 -- 172.21.15.123:0/1755461951 reap_dead start 2024-04-25T19:08:34.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.574+0000 7f4497400640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c003a00 0x7f447c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.574+0000 7f4496a00640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 0x7f447c004c10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f448400c0e0 tx=0x7f447c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1786a5cbb3c3aac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.575+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484007c20 con 0x7f447c004800 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.575+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4484005af0 con 0x7f447c004800 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.575+0000 7f4496000640 1 -- 172.21.15.123:0/1755461951 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484007070 con 0x7f447c004800 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.575+0000 7f449d000640 1 -- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 msgr2=0x7f447c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.575+0000 7f449d000640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 0x7f447c004c10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f448400c0e0 tx=0x7f447c005830 comp rx=0 tx=0).stop 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.576+0000 7f449d000640 1 -- 172.21.15.123:0/1755461951 shutdown_connections 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.576+0000 7f449d000640 1 --2- 172.21.15.123:0/1755461951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c004800 0x7f447c004c10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.576+0000 7f449d000640 1 -- 172.21.15.123:0/1755461951 >> 172.21.15.123:0/1755461951 conn(0x7f4498073200 msgr2=0x7f4498073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:34.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.576+0000 7f449d000640 1 -- 172.21.15.123:0/1755461951 shutdown_connections 2024-04-25T19:08:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.576+0000 7f449d000640 1 -- 172.21.15.123:0/1755461951 wait complete. 2024-04-25T19:08:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.578+0000 7f449d000640 1 Processor -- start 2024-04-25T19:08:34.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.578+0000 7f449d000640 1 -- start start 2024-04-25T19:08:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 0x7f4498146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44980cd8a0 0x7f44980cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f449815e880 con 0x7f449813f380 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f449815e700 con 0x7f447c004800 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f449d000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f449815ea00 con 0x7f44980cd8a0 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57934/0 (socket says 172.21.15.123:57934) 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 -- 172.21.15.123:0/616491256 learned_addr learned my addr 172.21.15.123:0/616491256 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4497e00640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44980cd8a0 0x7f44980cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4497400640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 0x7f4498146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 -- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44980cd8a0 msgr2=0x7f44980cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44980cd8a0 0x7f44980cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 -- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 msgr2=0x7f4498146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.580+0000 7f4496a00640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 0x7f4498146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f4496a00640 1 -- 172.21.15.123:0/616491256 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980ce1f0 con 0x7f449813f380 2024-04-25T19:08:34.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f4497400640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 0x7f4498146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:34.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f4496a00640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f448402ad20 tx=0x7f449806f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484037040 con 0x7f449813f380 2024-04-25T19:08:34.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4484041070 con 0x7f449813f380 2024-04-25T19:08:34.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484004a00 con 0x7f449813f380 2024-04-25T19:08:34.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981f2170 con 0x7f449813f380 2024-04-25T19:08:34.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.581+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449815c290 con 0x7f449813f380 2024-04-25T19:08:34.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.582+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449815bd20 con 0x7f449813f380 2024-04-25T19:08:34.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.584+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4484004ba0 con 0x7f449813f380 2024-04-25T19:08:34.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.584+0000 7f448f400640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 0x7f44780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:34.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.585+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f44840b4fd0 con 0x7f449813f380 2024-04-25T19:08:34.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.585+0000 7f4497400640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 0x7f44780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:34.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.585+0000 7f4497400640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 0x7f44780769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4498069e80 tx=0x7f4488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:34.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.590+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44980ce1f0 con 0x7f449813f380 2024-04-25T19:08:34.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.782+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4498050740 con 0x7f44780744e0 2024-04-25T19:08:34.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.784+0000 7f448f400640 1 -- 172.21.15.123:0/616491256 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7f4498050740 con 0x7f44780744e0 2024-04-25T19:08:34.787 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:34.787 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 msgr2=0x7f44780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 0x7f44780769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4498069e80 tx=0x7f4488008040 comp rx=0 tx=0).stop 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 msgr2=0x7f449815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f448402ad20 tx=0x7f449806f510 comp rx=0 tx=0).stop 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 shutdown_connections 2024-04-25T19:08:34.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f44780744e0 0x7f44780769f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44980cd8a0 0x7f44980cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449813f380 0x7f449815b6b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 --2- 172.21.15.123:0/616491256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c004800 0x7f4498146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 >> 172.21.15.123:0/616491256 conn(0x7f4498073200 msgr2=0x7f44981602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 shutdown_connections 2024-04-25T19:08:34.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:34.787+0000 7f449d000640 1 -- 172.21.15.123:0/616491256 wait complete. 2024-04-25T19:08:34.813 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-25T19:08:33.087064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225604,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749116,"statfs":{"total":773094113280,"available":772863094784,"internally_reserved":0,"allocated":5382144,"data_stored":2993480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617455,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-121,"num_write_kb":-663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.482392"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T19:08:29.974555+0000","last_change":"2024-04-25T19:02:29.662074+0000","last_active":"2024-04-25T19:08:29.974555+0000","last_peered":"2024-04-25T19:08:29.974555+0000","last_clean":"2024-04-25T19:08:29.974555+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:08:29.974555+0000","last_undegraded":"2024-04-25T19:08:29.974555+0000","last_fullsized":"2024-04-25T19:08:29.974555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:02:27.892762+0000","last_clean_scrub_stamp":"2024-04-25T19:02:27.892762+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:10.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:08:34.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:08:35.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.071+0000 7fe11e200640 1 Processor -- start 2024-04-25T19:08:35.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.071+0000 7fe11e200640 1 -- start start 2024-04-25T19:08:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.073+0000 7fe11e200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118161b10 0x7fe118161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.073+0000 7fe11e200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe118154950 0x7fe118154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.073+0000 7fe11e200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe118071600 con 0x7fe1180d1120 2024-04-25T19:08:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.073+0000 7fe11e200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe1180d1530 con 0x7fe118154950 2024-04-25T19:08:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.073+0000 7fe11e200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe118162460 con 0x7fe118161b10 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe118154950 0x7fe118154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe118154950 0x7fe118154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57938/0 (socket says 172.21.15.123:57938) 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 -- 172.21.15.123:0/807700942 learned_addr learned my addr 172.21.15.123:0/807700942 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe10fe00640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118161b10 0x7fe118161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 -- 172.21.15.123:0/807700942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118161b10 msgr2=0x7fe118161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118161b10 0x7fe118161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 -- 172.21.15.123:0/807700942 >> v1:172.21.15.146:6789/0 conn(0x7fe1180d1120 legacy=0x7fe118070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.074+0000 7fe117e00640 1 -- 172.21.15.123:0/807700942 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1180aec30 con 0x7fe118154950 2024-04-25T19:08:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.075+0000 7fe117e00640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118154950 0x7fe118154d60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe110009e50 tx=0x7fe11002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e537529011414ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.075+0000 7fe116a00640 1 -- 172.21.15.123:0/807700942 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110036070 con 0x7fe118154950 2024-04-25T19:08:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.075+0000 7fe116a00640 1 -- 172.21.15.123:0/807700942 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe110031600 con 0x7fe118154950 2024-04-25T19:08:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.075+0000 7fe116a00640 1 -- 172.21.15.123:0/807700942 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100319d0 con 0x7fe118154950 2024-04-25T19:08:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 -- 172.21.15.123:0/807700942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118154950 msgr2=0x7fe118154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118154950 0x7fe118154d60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe110009e50 tx=0x7fe11002a5b0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 -- 172.21.15.123:0/807700942 shutdown_connections 2024-04-25T19:08:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118154950 0x7fe118154d60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 --2- 172.21.15.123:0/807700942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118161b10 0x7fe118161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 -- 172.21.15.123:0/807700942 >> 172.21.15.123:0/807700942 conn(0x7fe1180d1ae0 msgr2=0x7fe1180d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.076+0000 7fe11e200640 1 -- 172.21.15.123:0/807700942 shutdown_connections 2024-04-25T19:08:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.077+0000 7fe11e200640 1 -- 172.21.15.123:0/807700942 wait complete. 2024-04-25T19:08:35.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.079+0000 7fe11e200640 1 Processor -- start 2024-04-25T19:08:35.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.079+0000 7fe11e200640 1 -- start start 2024-04-25T19:08:35.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118154950 0x7fe118155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe118161b10 0x7fe118155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180af4d0 con 0x7fe1180d1120 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180af1f0 con 0x7fe118161b10 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe11e200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1181552a0 con 0x7fe118154950 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe117400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe117400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57942/0 (socket says 172.21.15.123:57942) 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.081+0000 7fe117400640 1 -- 172.21.15.123:0/4163256413 learned_addr learned my addr 172.21.15.123:0/4163256413 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117e00640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe118161b10 0x7fe118155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe10fe00640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118154950 0x7fe118155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 -- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118154950 msgr2=0x7fe118155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118154950 0x7fe118155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 -- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe118161b10 msgr2=0x7fe118155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe118161b10 0x7fe118155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 -- 172.21.15.123:0/4163256413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1181e9370 con 0x7fe1180d1120 2024-04-25T19:08:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.082+0000 7fe117400640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe10800b3f0 tx=0x7fe10800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.083+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108015020 con 0x7fe1180d1120 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.083+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108011830 con 0x7fe1180d1120 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.083+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108011c20 con 0x7fe1180d1120 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.083+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118159bb0 con 0x7fe1180d1120 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.083+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11815a170 con 0x7fe1180d1120 2024-04-25T19:08:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.084+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe118076a40 con 0x7fe1180d1120 2024-04-25T19:08:35.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.085+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe108018020 con 0x7fe1180d1120 2024-04-25T19:08:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.087+0000 7fe10f400640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 0x7fe0f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.087+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fe108094d00 con 0x7fe1180d1120 2024-04-25T19:08:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.087+0000 7fe10fe00640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 0x7fe0f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.088+0000 7fe10fe00640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 0x7fe0f40769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1180d0ba0 tx=0x7fe100008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:35.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.092+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe10801f030 con 0x7fe1180d1120 2024-04-25T19:08:35.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.292+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe118070250 con 0x7fe1180d1120 2024-04-25T19:08:35.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.294+0000 7fe10f400640 1 -- 172.21.15.123:0/4163256413 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13343 (secure 0 0 0) 0x7fe1181e9370 con 0x7fe1180d1120 2024-04-25T19:08:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:35.296 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":45,"fsid":"a2d72801-e932-4284-9776-fce727cd5af1","created":"2024-04-25T19:02:13.912756+0000","modified":"2024-04-25T19:08:29.904272+0000","last_up_change":"2024-04-25T19:02:26.887000+0000","last_in_change":"2024-04-25T19:02:17.908495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:02:26.996878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45334943-7f6e-4b40-8809-22b648c39b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6817","nonce":1214013409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6819","nonce":1214013409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6823","nonce":1214013409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1214013409},{"type":"v1","addr":"172.21.15.123:6821","nonce":1214013409}]},"public_addr":"172.21.15.123:6817/1214013409","cluster_addr":"172.21.15.123:6819/1214013409","heartbeat_back_addr":"172.21.15.123:6823/1214013409","heartbeat_front_addr":"172.21.15.123:6821/1214013409","state":["exists","up"]},{"osd":1,"uuid":"d7e84f25-5ec3-4e15-9d9d-2208c882c785","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6809","nonce":1658520408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6811","nonce":1658520408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6815","nonce":1658520408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1658520408},{"type":"v1","addr":"172.21.15.123:6813","nonce":1658520408}]},"public_addr":"172.21.15.123:6809/1658520408","cluster_addr":"172.21.15.123:6811/1658520408","heartbeat_back_addr":"172.21.15.123:6815/1658520408","heartbeat_front_addr":"172.21.15.123:6813/1658520408","state":["exists","up"]},{"osd":2,"uuid":"61e5a24c-4380-4098-8dce-72bf1dff941b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6801","nonce":3468692589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6803","nonce":3468692589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6808","nonce":3468692589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3468692589},{"type":"v1","addr":"172.21.15.123:6805","nonce":3468692589}]},"public_addr":"172.21.15.123:6801/3468692589","cluster_addr":"172.21.15.123:6803/3468692589","heartbeat_back_addr":"172.21.15.123:6808/3468692589","heartbeat_front_addr":"172.21.15.123:6805/3468692589","state":["exists","up"]},{"osd":3,"uuid":"031b0f37-d4ee-4e43-8a3a-453a94c52f65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6825","nonce":3224162034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6827","nonce":3224162034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6831","nonce":3224162034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3224162034},{"type":"v1","addr":"172.21.15.123:6829","nonce":3224162034}]},"public_addr":"172.21.15.123:6825/3224162034","cluster_addr":"172.21.15.123:6827/3224162034","heartbeat_back_addr":"172.21.15.123:6831/3224162034","heartbeat_front_addr":"172.21.15.123:6829/3224162034","state":["exists","up"]},{"osd":4,"uuid":"0d80edc9-5cee-4293-9dcb-e78b49c9ee0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6801","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6803","nonce":1889014083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6805","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6807","nonce":1889014083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6813","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6815","nonce":1889014083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1889014083},{"type":"v1","addr":"172.21.15.146:6811","nonce":1889014083}]},"public_addr":"172.21.15.146:6803/1889014083","cluster_addr":"172.21.15.146:6807/1889014083","heartbeat_back_addr":"172.21.15.146:6815/1889014083","heartbeat_front_addr":"172.21.15.146:6811/1889014083","state":["exists","up"]},{"osd":5,"uuid":"18466f32-ab3c-49ce-80ce-e298e4e0a0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6802","nonce":2028526652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6806","nonce":2028526652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6814","nonce":2028526652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2028526652},{"type":"v1","addr":"172.21.15.146:6810","nonce":2028526652}]},"public_addr":"172.21.15.146:6802/2028526652","cluster_addr":"172.21.15.146:6806/2028526652","heartbeat_back_addr":"172.21.15.146:6814/2028526652","heartbeat_front_addr":"172.21.15.146:6810/2028526652","state":["exists","up"]},{"osd":6,"uuid":"87be9fd7-46a7-45f4-afa8-d04439198a81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6825","nonce":510483223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6827","nonce":510483223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6831","nonce":510483223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":510483223},{"type":"v1","addr":"172.21.15.146:6829","nonce":510483223}]},"public_addr":"172.21.15.146:6825/510483223","cluster_addr":"172.21.15.146:6827/510483223","heartbeat_back_addr":"172.21.15.146:6831/510483223","heartbeat_front_addr":"172.21.15.146:6829/510483223","state":["exists","up"]},{"osd":7,"uuid":"ad614261-aee1-4cc1-83b5-874100e98720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6817","nonce":3816144943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6819","nonce":3816144943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6823","nonce":3816144943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":3816144943},{"type":"v1","addr":"172.21.15.146:6821","nonce":3816144943}]},"public_addr":"172.21.15.146:6817/3816144943","cluster_addr":"172.21.15.146:6819/3816144943","heartbeat_back_addr":"172.21.15.146:6823/3816144943","heartbeat_front_addr":"172.21.15.146:6821/3816144943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.447477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.607161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.063087+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.392809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.469557+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.606879+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:22.949816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:02:23.635936+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6835/4124573423":"2024-04-26T19:08:26.599316+0000","172.21.15.123:6834/4124573423":"2024-04-26T19:08:26.599316+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:08:35.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.296+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 msgr2=0x7fe0f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:35.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.296+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 0x7fe0f40769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1180d0ba0 tx=0x7fe100008040 comp rx=0 tx=0).stop 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 msgr2=0x7fe1181e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe10800b3f0 tx=0x7fe10800b8b0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 shutdown_connections 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fe0f40744b0 0x7fe0f40769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe118161b10 0x7fe118155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe118154950 0x7fe118155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 --2- 172.21.15.123:0/4163256413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1180d1120 0x7fe1181e8e30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:35.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 >> 172.21.15.123:0/4163256413 conn(0x7fe1180d1ae0 msgr2=0x7fe1180b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:35.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 shutdown_connections 2024-04-25T19:08:35.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:35.297+0000 7fe11e200640 1 -- 172.21.15.123:0/4163256413 wait complete. 2024-04-25T19:08:36.328 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T19:08:36.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:36.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.614+0000 7f7343e00640 1 Processor -- start 2024-04-25T19:08:36.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.615+0000 7f7343e00640 1 -- start start 2024-04-25T19:08:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.617+0000 7f7343e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73440d4060 0x7f73440b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.617+0000 7f7343e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.617+0000 7f7343e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73440d53f0 con 0x7f734406d330 2024-04-25T19:08:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.617+0000 7f7343e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f734406c570 con 0x7f73440d4060 2024-04-25T19:08:36.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.617+0000 7f7343e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f734406c850 con 0x7f73440d07f0 2024-04-25T19:08:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.618+0000 7f7343400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.618+0000 7f7343400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:48272/0 (socket says 172.21.15.123:48272) 2024-04-25T19:08:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.618+0000 7f7343400640 1 -- 172.21.15.123:0/3069777438 learned_addr learned my addr 172.21.15.123:0/3069777438 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:36.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.618+0000 7f7342a00640 1 --2- 172.21.15.123:0/3069777438 >> v2:172.21.15.146:3300/0 conn(0x7f73440d4060 0x7f73440b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.619+0000 7f7343400640 1 -- 172.21.15.123:0/3069777438 >> v2:172.21.15.146:3300/0 conn(0x7f73440d4060 msgr2=0x7f73440b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:36.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.619+0000 7f7343400640 1 --2- 172.21.15.123:0/3069777438 >> v2:172.21.15.146:3300/0 conn(0x7f73440d4060 0x7f73440b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.619+0000 7f7343400640 1 -- 172.21.15.123:0/3069777438 >> v1:172.21.15.146:6789/0 conn(0x7f734406d330 legacy=0x7f734406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.619+0000 7f7343400640 1 -- 172.21.15.123:0/3069777438 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734406cb30 con 0x7f73440d07f0 2024-04-25T19:08:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.620+0000 7f7343400640 1 --2- 172.21.15.123:0/3069777438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f733c009e50 tx=0x7f733c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e183229b4dd78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.620+0000 7f7342000640 1 -- 172.21.15.123:0/3069777438 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c036070 con 0x7f73440d07f0 2024-04-25T19:08:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7342000640 1 -- 172.21.15.123:0/3069777438 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c031600 con 0x7f73440d07f0 2024-04-25T19:08:36.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7342000640 1 -- 172.21.15.123:0/3069777438 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c0319d0 con 0x7f73440d07f0 2024-04-25T19:08:36.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 -- 172.21.15.123:0/3069777438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 msgr2=0x7f73440d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 --2- 172.21.15.123:0/3069777438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f733c009e50 tx=0x7f733c02a710 comp rx=0 tx=0).stop 2024-04-25T19:08:36.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 -- 172.21.15.123:0/3069777438 shutdown_connections 2024-04-25T19:08:36.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 --2- 172.21.15.123:0/3069777438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73440d07f0 0x7f73440d0c00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 --2- 172.21.15.123:0/3069777438 >> v2:172.21.15.146:3300/0 conn(0x7f73440d4060 0x7f73440b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.621+0000 7f7343e00640 1 -- 172.21.15.123:0/3069777438 >> 172.21.15.123:0/3069777438 conn(0x7f73440b74a0 msgr2=0x7f73440b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:36.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.622+0000 7f7343e00640 1 -- 172.21.15.123:0/3069777438 shutdown_connections 2024-04-25T19:08:36.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.622+0000 7f7343e00640 1 -- 172.21.15.123:0/3069777438 wait complete. 2024-04-25T19:08:36.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.624+0000 7f7343e00640 1 Processor -- start 2024-04-25T19:08:36.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.624+0000 7f7343e00640 1 -- start start 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734406d330 0x7f73440c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 0x7f73441f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73440d4060 0x7f73441eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73440b9470 con 0x7f73440d07f0 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f734406de50 con 0x7f73440d4060 2024-04-25T19:08:36.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7344160340 con 0x7f734406d330 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 0x7f73441f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 0x7f73441f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48016/0 (socket says 172.21.15.123:48016) 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 -- 172.21.15.123:0/181703643 learned_addr learned my addr 172.21.15.123:0/181703643 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7343400640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73440d4060 0x7f73441eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f7342a00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734406d330 0x7f73440c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 -- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734406d330 msgr2=0x7f73440c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734406d330 0x7f73440c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 -- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73440d4060 msgr2=0x7f73441eb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.626+0000 7f733a000640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73440d4060 0x7f73441eb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f733a000640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734415bfc0 con 0x7f73440d07f0 2024-04-25T19:08:36.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f733a000640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 0x7f73441f61d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c3c0 tx=0x7f732c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0052d0 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c005490 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c007260 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73441ea450 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.627+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441eaa10 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.628+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7344050740 con 0x7f73440d07f0 2024-04-25T19:08:36.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.630+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f732c007400 con 0x7f73440d07f0 2024-04-25T19:08:36.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f733be00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73200744e0 0x7f73200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f732c0945f0 con 0x7f73440d07f0 2024-04-25T19:08:36.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f733be00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f732007a9d0 0x7f732007ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f732007d530 con 0x7f732007a9d0 2024-04-25T19:08:36.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f732c09b050 con 0x7f73440d07f0 2024-04-25T19:08:36.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f7342a00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73200744e0 0x7f73200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.631+0000 7f7343400640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f732007a9d0 0x7f732007ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.632+0000 7f7342a00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73200744e0 0x7f73200769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7334009a30 tx=0x7f7334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.632+0000 7f7343400640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f732007a9d0 0x7f732007ce60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.633+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== osd.0 v2:172.21.15.123:6816/1214013409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732007d530 con 0x7f732007a9d0 2024-04-25T19:08:36.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.648+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 --> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f73441ea1c0 con 0x7f732007a9d0 2024-04-25T19:08:36.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.648+0000 7f733be00640 1 -- 172.21.15.123:0/181703643 <== osd.0 v2:172.21.15.123:6816/1214013409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f73441ea1c0 con 0x7f732007a9d0 2024-04-25T19:08:36.650 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:36.650 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:36.650 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:36.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f732007a9d0 msgr2=0x7f732007ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6816/1214013409,v1:172.21.15.123:6817/1214013409] conn(0x7f732007a9d0 0x7f732007ce60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73200744e0 msgr2=0x7f73200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f73200744e0 0x7f73200769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7334009a30 tx=0x7f7334008040 comp rx=0 tx=0).stop 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 msgr2=0x7f73441f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 --2- 172.21.15.123:0/181703643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73440d07f0 0x7f73441f61d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c3c0 tx=0x7f732c00c880 comp rx=0 tx=0).stop 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343400640 1 -- 172.21.15.123:0/181703643 reap_dead start 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 shutdown_connections 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 >> 172.21.15.123:0/181703643 conn(0x7f73440b74a0 msgr2=0x7f73440b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 shutdown_connections 2024-04-25T19:08:36.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.649+0000 7f7343e00640 1 -- 172.21.15.123:0/181703643 wait complete. 2024-04-25T19:08:36.676 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T19:08:36.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.947+0000 7f25b3200640 1 Processor -- start 2024-04-25T19:08:36.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.947+0000 7f25b3200640 1 -- start start 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b3200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25b406d580 0x7f25b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25b415e2f0 0x7f25b415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b406ded0 con 0x7f25b406fe20 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f25b415f070 con 0x7f25b415e2f0 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b3200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f25b406f340 con 0x7f25b406d580 2024-04-25T19:08:36.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b1e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f25b406fe20 0x7f25b415b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56412/0 (socket says 172.21.15.123:56412) 2024-04-25T19:08:36.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b1e00640 1 -- 172.21.15.123:0/2454364474 learned_addr learned my addr 172.21.15.123:0/2454364474 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:36.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.949+0000 7f25b2800640 1 --2- 172.21.15.123:0/2454364474 >> v2:172.21.15.123:3300/0 conn(0x7f25b415e2f0 0x7f25b415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b1400640 1 --2- 172.21.15.123:0/2454364474 >> v2:172.21.15.146:3300/0 conn(0x7f25b406d580 0x7f25b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b0a00640 1 -- 172.21.15.123:0/2454364474 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437682044 0 0) 0x7f25a4001110 con 0x7f25b406fe20 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b0a00640 1 -- 172.21.15.123:0/2454364474 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2598003520 con 0x7f25b406fe20 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b2800640 1 -- 172.21.15.123:0/2454364474 >> v2:172.21.15.146:3300/0 conn(0x7f25b406d580 msgr2=0x7f25b406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b2800640 1 --2- 172.21.15.123:0/2454364474 >> v2:172.21.15.146:3300/0 conn(0x7f25b406d580 0x7f25b406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b2800640 1 -- 172.21.15.123:0/2454364474 >> v1:172.21.15.123:6789/0 conn(0x7f25b406fe20 legacy=0x7f25b415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.950+0000 7f25b2800640 1 -- 172.21.15.123:0/2454364474 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b406f620 con 0x7f25b415e2f0 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.951+0000 7f25b2800640 1 --2- 172.21.15.123:0/2454364474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b415e2f0 0x7f25b415e700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f25b4070260 tx=0x7f25ac02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67b7b614fe4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.952+0000 7f25b0a00640 1 -- 172.21.15.123:0/2454364474 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac007740 con 0x7f25b415e2f0 2024-04-25T19:08:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.952+0000 7f25b0a00640 1 -- 172.21.15.123:0/2454364474 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ac0078e0 con 0x7f25b415e2f0 2024-04-25T19:08:36.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.952+0000 7f25b0a00640 1 -- 172.21.15.123:0/2454364474 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac007cb0 con 0x7f25b415e2f0 2024-04-25T19:08:36.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.952+0000 7f25b3200640 1 -- 172.21.15.123:0/2454364474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b415e2f0 msgr2=0x7f25b415e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.952+0000 7f25b3200640 1 --2- 172.21.15.123:0/2454364474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b415e2f0 0x7f25b415e700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f25b4070260 tx=0x7f25ac02a5f0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 -- 172.21.15.123:0/2454364474 shutdown_connections 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 --2- 172.21.15.123:0/2454364474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b415e2f0 0x7f25b415e700 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 --2- 172.21.15.123:0/2454364474 >> v2:172.21.15.146:3300/0 conn(0x7f25b406d580 0x7f25b406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 -- 172.21.15.123:0/2454364474 >> 172.21.15.123:0/2454364474 conn(0x7f25b4073200 msgr2=0x7f25b4073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 -- 172.21.15.123:0/2454364474 shutdown_connections 2024-04-25T19:08:36.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.953+0000 7f25b3200640 1 -- 172.21.15.123:0/2454364474 wait complete. 2024-04-25T19:08:36.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.955+0000 7f25b3200640 1 Processor -- start 2024-04-25T19:08:36.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.955+0000 7f25b3200640 1 -- start start 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25b406fe20 0x7f25b41371f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25b415e2f0 0x7f25b41ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25b415edc0 con 0x7f25b406d580 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25b415ec40 con 0x7f25b415e2f0 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b3200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f25b40ce500 con 0x7f25b406fe20 2024-04-25T19:08:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48042/0 (socket says 172.21.15.123:48042) 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 -- 172.21.15.123:0/1192094426 learned_addr learned my addr 172.21.15.123:0/1192094426 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 -- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25b406fe20 msgr2=0x7f25b41371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25b406fe20 0x7f25b41371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 -- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25b415e2f0 msgr2=0x7f25b41ec2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25b415e2f0 0x7f25b41ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.957+0000 7f25b1e00640 1 -- 172.21.15.123:0/1192094426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41ec7e0 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25b1e00640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c340 tx=0x7f25a400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4017070 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a4005ce0 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b4134d80 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a40072f0 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.958+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b4135340 con 0x7f25b406d580 2024-04-25T19:08:36.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.959+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b40b3120 con 0x7f25b406d580 2024-04-25T19:08:36.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.960+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25a401a070 con 0x7f25b406d580 2024-04-25T19:08:36.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.962+0000 7f25aa000640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 0x7f25940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:36.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.962+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f25a4094560 con 0x7f25b406d580 2024-04-25T19:08:36.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.962+0000 7f25b1400640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 0x7f25940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:36.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.963+0000 7f25b1400640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 0x7f25940769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:36.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:36.967+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25b406ded0 con 0x7f25b406d580 2024-04-25T19:08:37.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.168+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f25b40d11d0 con 0x7f25940744b0 2024-04-25T19:08:37.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.169+0000 7f25aa000640 1 -- 172.21.15.123:0/1192094426 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f25b40d11d0 con 0x7f25940744b0 2024-04-25T19:08:37.171 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 msgr2=0x7f25940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 0x7f25940769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 msgr2=0x7f25b41345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c340 tx=0x7f25a400c800 comp rx=0 tx=0).stop 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 shutdown_connections 2024-04-25T19:08:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f25940744b0 0x7f25940769c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25b415e2f0 0x7f25b41ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25b406fe20 0x7f25b41371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.171+0000 7f25b3200640 1 --2- 172.21.15.123:0/1192094426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b406d580 0x7f25b41345c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.172+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 >> 172.21.15.123:0/1192094426 conn(0x7f25b4073200 msgr2=0x7f25b415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.172+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 shutdown_connections 2024-04-25T19:08:37.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.172+0000 7f25b3200640 1 -- 172.21.15.123:0/1192094426 wait complete. 2024-04-25T19:08:37.196 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T19:08:37.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:37.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.466+0000 7f2c72800640 1 Processor -- start 2024-04-25T19:08:37.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.466+0000 7f2c72800640 1 -- start start 2024-04-25T19:08:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c72800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 0x7f2c6c0d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c72800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c6c0d3040 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c72800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c6c162840 con 0x7f2c6c0ce740 2024-04-25T19:08:37.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c72800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6c0d2670 con 0x7f2c6c162430 2024-04-25T19:08:37.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c71400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2c6c0b2d30 0x7f2c6c0b5280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56420/0 (socket says 172.21.15.123:56420) 2024-04-25T19:08:37.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c71400640 1 -- 172.21.15.123:0/317108728 learned_addr learned my addr 172.21.15.123:0/317108728 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:37.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.469+0000 7f2c70a00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 0x7f2c6c0d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073557907 0 0) 0x7f2c6c0d3040 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c50003520 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 912255249 0 0) 0x7f2c50003520 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 msgr2=0x7f2c6c0d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 0x7f2c6c0d2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 >> v1:172.21.15.146:6789/0 conn(0x7f2c6c0ce740 legacy=0x7f2c6c0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c0b5a40 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f2c5c001670 con 0x7f2c6c0b2d30 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 >> v1:172.21.15.123:6789/0 conn(0x7f2c6c0b2d30 legacy=0x7f2c6c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c50003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c50004800 0x7f2c50004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 0x7f2c5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c0d2670 con 0x7f2c500039e0 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500054e0 con 0x7f2c50004800 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.470+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5000bf00 con 0x7f2c6c162430 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c50003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c70a00640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c50004800 0x7f2c50004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 -- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 msgr2=0x7f2c5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c162430 0x7f2c5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 -- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c50004800 msgr2=0x7f2c50004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c50004800 0x7f2c50004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.471+0000 7f2c71400640 1 -- 172.21.15.123:0/317108728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50003520 con 0x7f2c500039e0 2024-04-25T19:08:37.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.472+0000 7f2c71400640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c50003e50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c160 tx=0x7f2c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7d9c6fcd5ffc0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c007c60 con 0x7f2c500039e0 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5c005af0 con 0x7f2c500039e0 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c6be00640 1 -- 172.21.15.123:0/317108728 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c007070 con 0x7f2c500039e0 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c72800640 1 -- 172.21.15.123:0/317108728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 msgr2=0x7f2c50003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c72800640 1 --2- 172.21.15.123:0/317108728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c50003e50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c160 tx=0x7f2c50005830 comp rx=0 tx=0).stop 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c71e00640 1 -- 172.21.15.123:0/317108728 reap_dead start 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c72800640 1 -- 172.21.15.123:0/317108728 shutdown_connections 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.473+0000 7f2c72800640 1 -- 172.21.15.123:0/317108728 >> 172.21.15.123:0/317108728 conn(0x7f2c6c0d0280 msgr2=0x7f2c6c0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.474+0000 7f2c72800640 1 -- 172.21.15.123:0/317108728 shutdown_connections 2024-04-25T19:08:37.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.474+0000 7f2c72800640 1 -- 172.21.15.123:0/317108728 wait complete. 2024-04-25T19:08:37.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.476+0000 7f2c72800640 1 Processor -- start 2024-04-25T19:08:37.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.476+0000 7f2c72800640 1 -- start start 2024-04-25T19:08:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c6c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c6c1e96c0 0x7f2c6c1e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c157240 0x7f2c6c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c0d2300 con 0x7f2c500039e0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c0d2020 con 0x7f2c6c1e96c0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c72800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6c0d2b00 con 0x7f2c6c157240 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c6c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c6c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48054/0 (socket says 172.21.15.123:48054) 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71400640 1 -- 172.21.15.123:0/1746485200 learned_addr learned my addr 172.21.15.123:0/1746485200 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c70a00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c6c1e96c0 0x7f2c6c1e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71e00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c157240 0x7f2c6c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71400640 1 -- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c157240 msgr2=0x7f2c6c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.478+0000 7f2c71400640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c6c157240 0x7f2c6c157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c71400640 1 -- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c6c1e96c0 msgr2=0x7f2c6c1e9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c71400640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c6c1e96c0 0x7f2c6c1e9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c71400640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c157b90 con 0x7f2c500039e0 2024-04-25T19:08:37.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c71400640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c6c0ccac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02ad30 tx=0x7f2c5c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c0015b0 con 0x7f2c500039e0 2024-04-25T19:08:37.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5c042070 con 0x7f2c500039e0 2024-04-25T19:08:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c0072b0 con 0x7f2c500039e0 2024-04-25T19:08:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6c0cba20 con 0x7f2c500039e0 2024-04-25T19:08:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.479+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6c0cbfe0 con 0x7f2c500039e0 2024-04-25T19:08:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.480+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c6c050740 con 0x7f2c500039e0 2024-04-25T19:08:37.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.482+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c5c007450 con 0x7f2c500039e0 2024-04-25T19:08:37.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.482+0000 7f2c69600640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2c4c0744b0 0x7f2c4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f2c5c0b5270 con 0x7f2c500039e0 2024-04-25T19:08:37.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c69600640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7f2c4c07a9a0 0x7f2c4c07ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c5c02ce50 con 0x7f2c4c07a9a0 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c70a00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2c4c0744b0 0x7f2c4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2c5c0bc050 con 0x7f2c500039e0 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c71e00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7f2c4c07a9a0 0x7f2c4c07ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.483+0000 7f2c70a00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2c4c0744b0 0x7f2c4c0769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009160 tx=0x7f2c58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.484+0000 7f2c71e00640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7f2c4c07a9a0 0x7f2c4c07ce30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.484+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== osd.1 v2:172.21.15.123:6807/1658520408 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c5c02ce50 con 0x7f2c4c07a9a0 2024-04-25T19:08:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.499+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 --> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2c6c0d1b30 con 0x7f2c4c07a9a0 2024-04-25T19:08:37.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.499+0000 7f2c69600640 1 -- 172.21.15.123:0/1746485200 <== osd.1 v2:172.21.15.123:6807/1658520408 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2c6c0d1b30 con 0x7f2c4c07a9a0 2024-04-25T19:08:37.501 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:37.501 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:37.501 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:37.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7f2c4c07a9a0 msgr2=0x7f2c4c07ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6807/1658520408,v1:172.21.15.123:6809/1658520408] conn(0x7f2c4c07a9a0 0x7f2c4c07ce30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2c4c0744b0 msgr2=0x7f2c4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f2c4c0744b0 0x7f2c4c0769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009160 tx=0x7f2c58008040 comp rx=0 tx=0).stop 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 msgr2=0x7f2c6c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 --2- 172.21.15.123:0/1746485200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c500039e0 0x7f2c6c0ccac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02ad30 tx=0x7f2c5c001970 comp rx=0 tx=0).stop 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c71e00640 1 -- 172.21.15.123:0/1746485200 reap_dead start 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 shutdown_connections 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 >> 172.21.15.123:0/1746485200 conn(0x7f2c6c0d0280 msgr2=0x7f2c6c15f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 shutdown_connections 2024-04-25T19:08:37.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.500+0000 7f2c72800640 1 -- 172.21.15.123:0/1746485200 wait complete. 2024-04-25T19:08:37.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T19:08:37.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.785+0000 7f508de00640 1 Processor -- start 2024-04-25T19:08:37.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.785+0000 7f508de00640 1 -- start start 2024-04-25T19:08:37.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50880d1120 0x7f5088070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5088161b10 0x7f5088161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5088154950 0x7f5088154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f50880d1530 con 0x7f50880d1120 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5088162460 con 0x7f5088161b10 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.787+0000 7f508de00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f50880aec30 con 0x7f5088154950 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50880d1120 0x7f5088070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50880d1120 0x7f5088070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48060/0 (socket says 172.21.15.123:48060) 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 -- 172.21.15.123:0/2905551520 learned_addr learned my addr 172.21.15.123:0/2905551520 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:37.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 -- 172.21.15.123:0/2905551520 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5088154950 msgr2=0x7f5088154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 --2- 172.21.15.123:0/2905551520 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5088154950 0x7f5088154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 -- 172.21.15.123:0/2905551520 >> v2:172.21.15.146:3300/0 conn(0x7f5088161b10 msgr2=0x7f5088161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 --2- 172.21.15.123:0/2905551520 >> v2:172.21.15.146:3300/0 conn(0x7f5088161b10 0x7f5088161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.788+0000 7f5086a00640 1 -- 172.21.15.123:0/2905551520 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50880aef10 con 0x7f50880d1120 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.789+0000 7f5086a00640 1 --2- 172.21.15.123:0/2905551520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50880d1120 0x7f5088070ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f50880b1260 tx=0x7f507802a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e142b600412947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.790+0000 7f5085600640 1 -- 172.21.15.123:0/2905551520 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078037070 con 0x7f50880d1120 2024-04-25T19:08:37.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.790+0000 7f5085600640 1 -- 172.21.15.123:0/2905551520 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507802ae20 con 0x7f50880d1120 2024-04-25T19:08:37.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.790+0000 7f5085600640 1 -- 172.21.15.123:0/2905551520 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078032660 con 0x7f50880d1120 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.790+0000 7f508de00640 1 -- 172.21.15.123:0/2905551520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50880d1120 msgr2=0x7f5088070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.790+0000 7f508de00640 1 --2- 172.21.15.123:0/2905551520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50880d1120 0x7f5088070ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f50880b1260 tx=0x7f507802a0d0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 -- 172.21.15.123:0/2905551520 shutdown_connections 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 --2- 172.21.15.123:0/2905551520 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5088154950 0x7f5088154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 --2- 172.21.15.123:0/2905551520 >> v2:172.21.15.146:3300/0 conn(0x7f5088161b10 0x7f5088161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 --2- 172.21.15.123:0/2905551520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50880d1120 0x7f5088070ef0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 -- 172.21.15.123:0/2905551520 >> 172.21.15.123:0/2905551520 conn(0x7f50880d1ae0 msgr2=0x7f50880d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 -- 172.21.15.123:0/2905551520 shutdown_connections 2024-04-25T19:08:37.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.791+0000 7f508de00640 1 -- 172.21.15.123:0/2905551520 wait complete. 2024-04-25T19:08:37.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.793+0000 7f508de00640 1 Processor -- start 2024-04-25T19:08:37.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.793+0000 7f508de00640 1 -- start start 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50880d1120 0x7f5088147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 0x7f5088157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50881552a0 con 0x7f5088154950 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50880af4d0 con 0x7f5088161b10 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.794+0000 7f508de00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5088144c10 con 0x7f50880d1120 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48072/0 (socket says 172.21.15.123:48072) 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 -- 172.21.15.123:0/813952041 learned_addr learned my addr 172.21.15.123:0/813952041 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086a00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50880d1120 0x7f5088147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5087400640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 0x7f5088157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 -- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50880d1120 msgr2=0x7f5088147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50880d1120 0x7f5088147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 -- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 msgr2=0x7f5088157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 0x7f5088157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5086000640 1 -- 172.21.15.123:0/813952041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508815b700 con 0x7f5088154950 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.795+0000 7f5087400640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 0x7f5088157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f5086000640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f507000b440 tx=0x7f507000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070015020 con 0x7f5088154950 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070011890 con 0x7f5088154950 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070011c80 con 0x7f5088154950 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881e7ad0 con 0x7f5088154950 2024-04-25T19:08:37.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.796+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881e8090 con 0x7f5088154950 2024-04-25T19:08:37.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.797+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5088076a40 con 0x7f5088154950 2024-04-25T19:08:37.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.798+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5070018020 con 0x7f5088154950 2024-04-25T19:08:37.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.799+0000 7f507f400640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 0x7f50640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:37.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.799+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f50700951e0 con 0x7f5088154950 2024-04-25T19:08:37.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.800+0000 7f5086a00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 0x7f50640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:37.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.800+0000 7f5086a00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 0x7f50640769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5078034000 tx=0x7f507802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.804+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f507001f020 con 0x7f5088154950 2024-04-25T19:08:37.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.994+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f508815b490 con 0x7f50640744b0 2024-04-25T19:08:37.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.995+0000 7f507f400640 1 -- 172.21.15.123:0/813952041 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f508815b490 con 0x7f50640744b0 2024-04-25T19:08:37.997 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 msgr2=0x7f50640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 0x7f50640769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5078034000 tx=0x7f507802e000 comp rx=0 tx=0).stop 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 msgr2=0x7f508815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f507000b440 tx=0x7f507000b900 comp rx=0 tx=0).stop 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 shutdown_connections 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f50640744b0 0x7f50640769c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5088161b10 0x7f5088157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5088154950 0x7f508815a9d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 --2- 172.21.15.123:0/813952041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50880d1120 0x7f5088147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 >> 172.21.15.123:0/813952041 conn(0x7f50880d1ae0 msgr2=0x7f50880b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:38.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 shutdown_connections 2024-04-25T19:08:38.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:37.998+0000 7f508de00640 1 -- 172.21.15.123:0/813952041 wait complete. 2024-04-25T19:08:38.023 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T19:08:38.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.297+0000 7f57bce00640 1 Processor -- start 2024-04-25T19:08:38.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.297+0000 7f57bce00640 1 -- start start 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57bce00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57b8156d70 0x7f57b81600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57bce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57b8070740 0x7f57b8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57bce00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b815cb00 con 0x7f57b806d580 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57bce00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f57b80d1410 con 0x7f57b8070740 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57bce00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f57b806f5e0 con 0x7f57b8156d70 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57b6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57b8070740 0x7f57b8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57b6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57b8070740 0x7f57b8070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48080/0 (socket says 172.21.15.123:48080) 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57b7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f57b806d580 0x7f57b806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56430/0 (socket says 172.21.15.123:56430) 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57b6a00640 1 -- 172.21.15.123:0/1118478161 learned_addr learned my addr 172.21.15.123:0/1118478161 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:38.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.300+0000 7f57b7400640 1 --2- 172.21.15.123:0/1118478161 >> v2:172.21.15.146:3300/0 conn(0x7f57b8156d70 0x7f57b81600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.301+0000 7f57b6a00640 1 -- 172.21.15.123:0/1118478161 >> v2:172.21.15.146:3300/0 conn(0x7f57b8156d70 msgr2=0x7f57b81600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.301+0000 7f57b6a00640 1 --2- 172.21.15.123:0/1118478161 >> v2:172.21.15.146:3300/0 conn(0x7f57b8156d70 0x7f57b81600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.301+0000 7f57b6a00640 1 -- 172.21.15.123:0/1118478161 >> v1:172.21.15.123:6789/0 conn(0x7f57b806d580 legacy=0x7f57b806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.301+0000 7f57b6a00640 1 -- 172.21.15.123:0/1118478161 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b806f8c0 con 0x7f57b8070740 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.302+0000 7f57b6a00640 1 --2- 172.21.15.123:0/1118478161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8070740 0x7f57b8070b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f57b80d27a0 tx=0x7f57a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc3c5a034baad37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.302+0000 7f57b6000640 1 -- 172.21.15.123:0/1118478161 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57b8070740 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.302+0000 7f57b6000640 1 -- 172.21.15.123:0/1118478161 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a4032440 con 0x7f57b8070740 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.302+0000 7f57b6000640 1 -- 172.21.15.123:0/1118478161 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4032810 con 0x7f57b8070740 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 -- 172.21.15.123:0/1118478161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8070740 msgr2=0x7f57b8070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 --2- 172.21.15.123:0/1118478161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8070740 0x7f57b8070b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f57b80d27a0 tx=0x7f57a402a270 comp rx=0 tx=0).stop 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 -- 172.21.15.123:0/1118478161 shutdown_connections 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 --2- 172.21.15.123:0/1118478161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8070740 0x7f57b8070b50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 --2- 172.21.15.123:0/1118478161 >> v2:172.21.15.146:3300/0 conn(0x7f57b8156d70 0x7f57b81600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 -- 172.21.15.123:0/1118478161 >> 172.21.15.123:0/1118478161 conn(0x7f57b8072fd0 msgr2=0x7f57b80733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 -- 172.21.15.123:0/1118478161 shutdown_connections 2024-04-25T19:08:38.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.303+0000 7f57bce00640 1 -- 172.21.15.123:0/1118478161 wait complete. 2024-04-25T19:08:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.305+0000 7f57bce00640 1 Processor -- start 2024-04-25T19:08:38.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.305+0000 7f57bce00640 1 -- start start 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b806d580 0x7f57b8150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8070740 0x7f57b8146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 0x7f57b815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8160630 con 0x7f57b8156d70 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57b806fe80 con 0x7f57b8070740 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57bce00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57b81607b0 con 0x7f57b806d580 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 0x7f57b815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57b7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 0x7f57b815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48082/0 (socket says 172.21.15.123:48082) 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.307+0000 7f57b7e00640 1 -- 172.21.15.123:0/4125908778 learned_addr learned my addr 172.21.15.123:0/4125908778 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b6a00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8070740 0x7f57b8146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7400640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b806d580 0x7f57b8150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 -- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b806d580 msgr2=0x7f57b8150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b806d580 0x7f57b8150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 -- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8070740 msgr2=0x7f57b8146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8070740 0x7f57b8146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8151830 con 0x7f57b8156d70 2024-04-25T19:08:38.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57b7e00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 0x7f57b815b590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c760 tx=0x7f57b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0017070 con 0x7f57b8156d70 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b000a640 con 0x7f57b8156d70 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.308+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b000a9f0 con 0x7f57b8156d70 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.309+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b8151b10 con 0x7f57b8156d70 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.309+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81520d0 con 0x7f57b8156d70 2024-04-25T19:08:38.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.310+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57b8070470 con 0x7f57b8156d70 2024-04-25T19:08:38.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.311+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57b001a070 con 0x7f57b8156d70 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57af400640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f57980744e0 0x7f57980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f57b0094630 con 0x7f57b8156d70 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57b7400640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f57980744e0 0x7f57980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57af400640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f579807a9d0 0x7f579807ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f579807d530 con 0x7f579807a9d0 2024-04-25T19:08:38.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.312+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f57b009b050 con 0x7f57b8156d70 2024-04-25T19:08:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.313+0000 7f57b6a00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f579807a9d0 0x7f579807ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.313+0000 7f57b7400640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f57980744e0 0x7f57980769f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b5740 tx=0x7f57a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.313+0000 7f57b6a00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f579807a9d0 0x7f579807ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.314+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== osd.2 v2:172.21.15.123:6800/3468692589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f579807d530 con 0x7f579807a9d0 2024-04-25T19:08:38.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.328+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 --> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57b81f6580 con 0x7f579807a9d0 2024-04-25T19:08:38.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57af400640 1 -- 172.21.15.123:0/4125908778 <== osd.2 v2:172.21.15.123:6800/3468692589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57b81f6580 con 0x7f579807a9d0 2024-04-25T19:08:38.331 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:38.331 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:38.331 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f579807a9d0 msgr2=0x7f579807ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6800/3468692589,v1:172.21.15.123:6801/3468692589] conn(0x7f579807a9d0 0x7f579807ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f57980744e0 msgr2=0x7f57980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f57980744e0 0x7f57980769f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b5740 tx=0x7f57a8009040 comp rx=0 tx=0).stop 2024-04-25T19:08:38.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 msgr2=0x7f57b815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 --2- 172.21.15.123:0/4125908778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b8156d70 0x7f57b815b590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c760 tx=0x7f57b000cc20 comp rx=0 tx=0).stop 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57b7e00640 1 -- 172.21.15.123:0/4125908778 reap_dead start 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 shutdown_connections 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.329+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 >> 172.21.15.123:0/4125908778 conn(0x7f57b8072fd0 msgr2=0x7f57b815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.330+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 shutdown_connections 2024-04-25T19:08:38.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.330+0000 7f57bce00640 1 -- 172.21.15.123:0/4125908778 wait complete. 2024-04-25T19:08:38.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T19:08:38.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.617+0000 7f504f200640 1 Processor -- start 2024-04-25T19:08:38.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.618+0000 7f504f200640 1 -- start start 2024-04-25T19:08:38.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504f200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f505006fe20 0x7f505015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f505015e2f0 0x7f505015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504f200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f505015f070 con 0x7f505006d580 2024-04-25T19:08:38.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f505006f340 con 0x7f505015e2f0 2024-04-25T19:08:38.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504f200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f505006f620 con 0x7f505006fe20 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f505015e2f0 0x7f505015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504e800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f505015e2f0 0x7f505015e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48092/0 (socket says 172.21.15.123:48092) 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.620+0000 7f504e800640 1 -- 172.21.15.123:0/3649276919 learned_addr learned my addr 172.21.15.123:0/3649276919 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.621+0000 7f504de00640 1 --2- 172.21.15.123:0/3649276919 >> v2:172.21.15.146:3300/0 conn(0x7f505006fe20 0x7f505015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.621+0000 7f504e800640 1 -- 172.21.15.123:0/3649276919 >> v2:172.21.15.146:3300/0 conn(0x7f505006fe20 msgr2=0x7f505015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.621+0000 7f504e800640 1 --2- 172.21.15.123:0/3649276919 >> v2:172.21.15.146:3300/0 conn(0x7f505006fe20 0x7f505015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.621+0000 7f504e800640 1 -- 172.21.15.123:0/3649276919 >> v1:172.21.15.146:6789/0 conn(0x7f505006d580 legacy=0x7f505006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.621+0000 7f504e800640 1 -- 172.21.15.123:0/3649276919 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505006f900 con 0x7f505015e2f0 2024-04-25T19:08:38.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.622+0000 7f504e800640 1 --2- 172.21.15.123:0/3649276919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505015e2f0 0x7f505015e700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5050074310 tx=0x7f504802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2c9f511a6d3106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.623+0000 7f504d400640 1 -- 172.21.15.123:0/3649276919 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048036070 con 0x7f505015e2f0 2024-04-25T19:08:38.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.623+0000 7f504d400640 1 -- 172.21.15.123:0/3649276919 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5048031600 con 0x7f505015e2f0 2024-04-25T19:08:38.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.623+0000 7f504d400640 1 -- 172.21.15.123:0/3649276919 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50480319d0 con 0x7f505015e2f0 2024-04-25T19:08:38.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.623+0000 7f504f200640 1 -- 172.21.15.123:0/3649276919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505015e2f0 msgr2=0x7f505015e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.623+0000 7f504f200640 1 --2- 172.21.15.123:0/3649276919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505015e2f0 0x7f505015e700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5050074310 tx=0x7f504802a6a0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 -- 172.21.15.123:0/3649276919 shutdown_connections 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 --2- 172.21.15.123:0/3649276919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505015e2f0 0x7f505015e700 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 --2- 172.21.15.123:0/3649276919 >> v2:172.21.15.146:3300/0 conn(0x7f505006fe20 0x7f505015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 -- 172.21.15.123:0/3649276919 >> 172.21.15.123:0/3649276919 conn(0x7f5050073200 msgr2=0x7f5050073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 -- 172.21.15.123:0/3649276919 shutdown_connections 2024-04-25T19:08:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.624+0000 7f504f200640 1 -- 172.21.15.123:0/3649276919 wait complete. 2024-04-25T19:08:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.626+0000 7f504f200640 1 Processor -- start 2024-04-25T19:08:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.626+0000 7f504f200640 1 -- start start 2024-04-25T19:08:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505006d580 0x7f50501f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505015e2f0 0x7f5050145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f505015edc0 con 0x7f505006fe20 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f505015ec40 con 0x7f505015e2f0 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f504f200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f50500ce500 con 0x7f505006d580 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48108/0 (socket says 172.21.15.123:48108) 2024-04-25T19:08:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 -- 172.21.15.123:0/1393035132 learned_addr learned my addr 172.21.15.123:0/1393035132 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 -- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505006d580 msgr2=0x7f50501f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505006d580 0x7f50501f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 -- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505015e2f0 msgr2=0x7f5050145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505015e2f0 0x7f5050145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.628+0000 7f5045400640 1 -- 172.21.15.123:0/1393035132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50501f5b20 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f5045400640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f503800c3f0 tx=0x7f503800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380052d0 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038005490 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038007260 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50501467d0 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.629+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5050144210 con 0x7f505006fe20 2024-04-25T19:08:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.630+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505015b740 con 0x7f505006fe20 2024-04-25T19:08:38.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.631+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5038007400 con 0x7f505006fe20 2024-04-25T19:08:38.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.632+0000 7f5046a00640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 0x7f502c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:38.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.632+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5038094450 con 0x7f505006fe20 2024-04-25T19:08:38.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.632+0000 7f504de00640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 0x7f502c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:38.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.633+0000 7f504de00640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 0x7f502c0769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5040006fd0 tx=0x7f5040008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:38.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.637+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f503805ee70 con 0x7f505006fe20 2024-04-25T19:08:38.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.835+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5050070a00 con 0x7f502c0744b0 2024-04-25T19:08:38.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.836+0000 7f5046a00640 1 -- 172.21.15.123:0/1393035132 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5050070a00 con 0x7f502c0744b0 2024-04-25T19:08:38.838 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T19:08:38.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 msgr2=0x7f502c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 0x7f502c0769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5040006fd0 tx=0x7f5040008040 comp rx=0 tx=0).stop 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 msgr2=0x7f5050161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f503800c3f0 tx=0x7f503800c8b0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 shutdown_connections 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f502c0744b0 0x7f502c0769c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505015e2f0 0x7f5050145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505006fe20 0x7f5050161df0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 --2- 172.21.15.123:0/1393035132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505006d580 0x7f50501f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:38.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 >> 172.21.15.123:0/1393035132 conn(0x7f5050073200 msgr2=0x7f5050160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:38.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 shutdown_connections 2024-04-25T19:08:38.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:38.839+0000 7f504f200640 1 -- 172.21.15.123:0/1393035132 wait complete. 2024-04-25T19:08:38.865 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T19:08:38.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:39.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.138+0000 7f420d600640 1 Processor -- start 2024-04-25T19:08:39.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.138+0000 7f420d600640 1 -- start start 2024-04-25T19:08:39.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.140+0000 7f420d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f4208160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.140+0000 7f420d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4208163fc0 con 0x7f420806d580 2024-04-25T19:08:39.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f420d600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42081642a0 con 0x7f4208156f00 2024-04-25T19:08:39.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f420d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f42080cf2a0 con 0x7f42080d0740 2024-04-25T19:08:39.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f41ffe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f420806d580 0x7f420806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56436/0 (socket says 172.21.15.123:56436) 2024-04-25T19:08:39.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f41ffe00640 1 -- 172.21.15.123:0/815180095 learned_addr learned my addr 172.21.15.123:0/815180095 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f420cc00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f4208160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3427990742 0 0) 0x7f4208163fc0 con 0x7f420806d580 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.141+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41ec003520 con 0x7f420806d580 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3011727415 0 0) 0x7f41ec003520 con 0x7f420806d580 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 msgr2=0x7f4208160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f4208160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 >> v1:172.21.15.146:6789/0 conn(0x7f4208156f00 legacy=0x7f4208070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4208164580 con 0x7f420806d580 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f41f0001670 con 0x7f420806d580 2024-04-25T19:08:39.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 >> v1:172.21.15.123:6789/0 conn(0x7f420806d580 legacy=0x7f420806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f41ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41ec004800 0x7f41ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f41ec0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42080cf2a0 con 0x7f41ec0039e0 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec00bb80 con 0x7f41ec004800 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec00bd80 con 0x7f42080d0740 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.142+0000 7f4207e00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f41ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f41ffe00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41ec004800 0x7f41ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f420cc00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f41ec0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f4207e00640 1 -- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 msgr2=0x7f41ec0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f4207e00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f41ec0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f4207e00640 1 -- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41ec004800 msgr2=0x7f41ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:39.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f4207e00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41ec004800 0x7f41ec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f4207e00640 1 -- 172.21.15.123:0/815180095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec003520 con 0x7f41ec0039e0 2024-04-25T19:08:39.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.143+0000 7f420cc00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42080d0740 0x7f41ec0052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.144+0000 7f4207e00640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f41ec003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f41f8001750 tx=0x7f41ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381d5c52bebbe5da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.144+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8001370 con 0x7f41ec0039e0 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.144+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f80052d0 con 0x7f41ec0039e0 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.144+0000 7f4207400640 1 -- 172.21.15.123:0/815180095 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f802abe0 con 0x7f41ec0039e0 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 -- 172.21.15.123:0/815180095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 msgr2=0x7f41ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 --2- 172.21.15.123:0/815180095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f41ec003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f41f8001750 tx=0x7f41ec005830 comp rx=0 tx=0).stop 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420cc00640 1 -- 172.21.15.123:0/815180095 reap_dead start 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 -- 172.21.15.123:0/815180095 shutdown_connections 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 -- 172.21.15.123:0/815180095 >> 172.21.15.123:0/815180095 conn(0x7f4208072fd0 msgr2=0x7f42080733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 -- 172.21.15.123:0/815180095 shutdown_connections 2024-04-25T19:08:39.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.145+0000 7f420d600640 1 -- 172.21.15.123:0/815180095 wait complete. 2024-04-25T19:08:39.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.147+0000 7f420d600640 1 Processor -- start 2024-04-25T19:08:39.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.148+0000 7f420d600640 1 -- start start 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f4208157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4208161dc0 0x7f42081621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4208162710 0x7f42080b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4208157490 con 0x7f41ec0039e0 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4208157310 con 0x7f4208162710 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f420d600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4208163b70 con 0x7f4208161dc0 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f4207e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f4208157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f4207e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f4208157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48122/0 (socket says 172.21.15.123:48122) 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.149+0000 7f4207e00640 1 -- 172.21.15.123:0/2233845866 learned_addr learned my addr 172.21.15.123:0/2233845866 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f41ffe00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4208161dc0 0x7f42081621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f420cc00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4208162710 0x7f42080b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 -- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4208161dc0 msgr2=0x7f42081621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4208161dc0 0x7f42081621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 -- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4208162710 msgr2=0x7f42080b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4208162710 0x7f42080b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42080b56a0 con 0x7f41ec0039e0 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.150+0000 7f4207e00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f4208157920 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f41f8001870 tx=0x7f41f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.151+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8037040 con 0x7f41ec0039e0 2024-04-25T19:08:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.151+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8005480 con 0x7f41ec0039e0 2024-04-25T19:08:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.151+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007a30 con 0x7f41ec0039e0 2024-04-25T19:08:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.151+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4208158530 con 0x7f41ec0039e0 2024-04-25T19:08:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.151+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42080b5c20 con 0x7f41ec0039e0 2024-04-25T19:08:39.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.152+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4208050740 con 0x7f41ec0039e0 2024-04-25T19:08:39.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.153+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f41f8044020 con 0x7f41ec0039e0 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f4204c00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f41e40744e0 0x7f41e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f41f80b58d0 con 0x7f41ec0039e0 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f4204c00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f41e407a9d0 0x7f41e407ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f41ffe00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f41e40744e0 0x7f41e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41f8037e50 con 0x7f41e407a9d0 2024-04-25T19:08:39.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.154+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41f80b5c80 con 0x7f41ec0039e0 2024-04-25T19:08:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.155+0000 7f420cc00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f41e407a9d0 0x7f41e407ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.155+0000 7f41ffe00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f41e40744e0 0x7f41e40769f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f41f0001050 tx=0x7f41f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.155+0000 7f420cc00640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f41e407a9d0 0x7f41e407ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.156+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== osd.3 v2:172.21.15.123:6824/3224162034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41f8037e50 con 0x7f41e407a9d0 2024-04-25T19:08:39.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.171+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 --> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4208073ee0 con 0x7f41e407a9d0 2024-04-25T19:08:39.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.171+0000 7f4204c00640 1 -- 172.21.15.123:0/2233845866 <== osd.3 v2:172.21.15.123:6824/3224162034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4208073ee0 con 0x7f41e407a9d0 2024-04-25T19:08:39.173 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:39.173 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f41e407a9d0 msgr2=0x7f41e407ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6824/3224162034,v1:172.21.15.123:6825/3224162034] conn(0x7f41e407a9d0 0x7f41e407ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f41e40744e0 msgr2=0x7f41e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f41e40744e0 0x7f41e40769f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f41f0001050 tx=0x7f41f0009040 comp rx=0 tx=0).stop 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 msgr2=0x7f4208157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 --2- 172.21.15.123:0/2233845866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ec0039e0 0x7f4208157920 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f41f8001870 tx=0x7f41f8001120 comp rx=0 tx=0).stop 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420cc00640 1 -- 172.21.15.123:0/2233845866 reap_dead start 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 shutdown_connections 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.172+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 >> 172.21.15.123:0/2233845866 conn(0x7f4208072fd0 msgr2=0x7f42080b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.173+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 shutdown_connections 2024-04-25T19:08:39.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.173+0000 7f420d600640 1 -- 172.21.15.123:0/2233845866 wait complete. 2024-04-25T19:08:39.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T19:08:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.467+0000 7fa2ace00640 1 Processor -- start 2024-04-25T19:08:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.467+0000 7fa2ace00640 1 -- start start 2024-04-25T19:08:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.469+0000 7fa2ace00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2a806fe20 0x7fa2a815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.469+0000 7fa2ace00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2a815e2f0 0x7fa2a815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.469+0000 7fa2ace00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a815f070 con 0x7fa2a806d580 2024-04-25T19:08:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.469+0000 7fa2ace00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a806f340 con 0x7fa2a815e2f0 2024-04-25T19:08:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.469+0000 7fa2ace00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a806f620 con 0x7fa2a806fe20 2024-04-25T19:08:39.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2a815e2f0 0x7fa2a815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a6a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa2a806d580 0x7fa2a806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56448/0 (socket says 172.21.15.123:56448) 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a6a00640 1 -- 172.21.15.123:0/557456672 learned_addr learned my addr 172.21.15.123:0/557456672 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a7400640 1 --2- 172.21.15.123:0/557456672 >> v2:172.21.15.146:3300/0 conn(0x7fa2a806fe20 0x7fa2a815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a6000640 1 -- 172.21.15.123:0/557456672 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730041660 0 0) 0x7fa2a815f070 con 0x7fa2a806d580 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.470+0000 7fa2a6000640 1 -- 172.21.15.123:0/557456672 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa28c003520 con 0x7fa2a806d580 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.471+0000 7fa2a7e00640 1 -- 172.21.15.123:0/557456672 >> v2:172.21.15.146:3300/0 conn(0x7fa2a806fe20 msgr2=0x7fa2a815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.471+0000 7fa2a7e00640 1 --2- 172.21.15.123:0/557456672 >> v2:172.21.15.146:3300/0 conn(0x7fa2a806fe20 0x7fa2a815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.471+0000 7fa2a7e00640 1 -- 172.21.15.123:0/557456672 >> v1:172.21.15.123:6789/0 conn(0x7fa2a806d580 legacy=0x7fa2a806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.471+0000 7fa2a7e00640 1 -- 172.21.15.123:0/557456672 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a806f900 con 0x7fa2a815e2f0 2024-04-25T19:08:39.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.472+0000 7fa2a7e00640 1 --2- 172.21.15.123:0/557456672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a815e2f0 0x7fa2a815e700 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8074310 tx=0x7fa2a002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14082bcf78c82eae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.472+0000 7fa2a6000640 1 -- 172.21.15.123:0/557456672 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0007740 con 0x7fa2a815e2f0 2024-04-25T19:08:39.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.472+0000 7fa2a6000640 1 -- 172.21.15.123:0/557456672 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a00078e0 con 0x7fa2a815e2f0 2024-04-25T19:08:39.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.472+0000 7fa2a6000640 1 -- 172.21.15.123:0/557456672 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0007cb0 con 0x7fa2a815e2f0 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 -- 172.21.15.123:0/557456672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a815e2f0 msgr2=0x7fa2a815e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 --2- 172.21.15.123:0/557456672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a815e2f0 0x7fa2a815e700 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8074310 tx=0x7fa2a002a5f0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 -- 172.21.15.123:0/557456672 shutdown_connections 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 --2- 172.21.15.123:0/557456672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a815e2f0 0x7fa2a815e700 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 --2- 172.21.15.123:0/557456672 >> v2:172.21.15.146:3300/0 conn(0x7fa2a806fe20 0x7fa2a815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 -- 172.21.15.123:0/557456672 >> 172.21.15.123:0/557456672 conn(0x7fa2a8073200 msgr2=0x7fa2a8073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 -- 172.21.15.123:0/557456672 shutdown_connections 2024-04-25T19:08:39.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.473+0000 7fa2ace00640 1 -- 172.21.15.123:0/557456672 wait complete. 2024-04-25T19:08:39.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.475+0000 7fa2ace00640 1 Processor -- start 2024-04-25T19:08:39.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.476+0000 7fa2ace00640 1 -- start start 2024-04-25T19:08:39.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a806d580 0x7fa2a8144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a815e2f0 0x7fa2a80cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a815edc0 con 0x7fa2a806fe20 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a815ec40 con 0x7fa2a806d580 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2ace00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80ce500 con 0x7fa2a815e2f0 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2a6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2a6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48152/0 (socket says 172.21.15.123:48152) 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.477+0000 7fa2a6a00640 1 -- 172.21.15.123:0/2076817917 learned_addr learned my addr 172.21.15.123:0/2076817917 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 -- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a815e2f0 msgr2=0x7fa2a80cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a815e2f0 0x7fa2a80cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 -- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a806d580 msgr2=0x7fa2a8144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a806d580 0x7fa2a8144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 -- 172.21.15.123:0/2076817917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8144770 con 0x7fa2a806fe20 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2a6a00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c390 tx=0x7fa29400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294017070 con 0x7fa2a806fe20 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa294005ce0 con 0x7fa2a806fe20 2024-04-25T19:08:39.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2940072f0 con 0x7fa2a806fe20 2024-04-25T19:08:39.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a80b51f0 con 0x7fa2a806fe20 2024-04-25T19:08:39.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.478+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a80b57b0 con 0x7fa2a806fe20 2024-04-25T19:08:39.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.480+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8070a00 con 0x7fa2a806fe20 2024-04-25T19:08:39.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.481+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa29401a070 con 0x7fa2a806fe20 2024-04-25T19:08:39.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.482+0000 7fa29f400640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 0x7fa2880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.482+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fa2940944e0 con 0x7fa2a806fe20 2024-04-25T19:08:39.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.482+0000 7fa2a7400640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 0x7fa2880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.483+0000 7fa2a7400640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 0x7fa2880769c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.487+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa29405edd0 con 0x7fa2a806fe20 2024-04-25T19:08:39.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.683+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa2a815ab20 con 0x7fa2880744b0 2024-04-25T19:08:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.684+0000 7fa29f400640 1 -- 172.21.15.123:0/2076817917 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa2a815ab20 con 0x7fa2880744b0 2024-04-25T19:08:39.685 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T19:08:39.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 msgr2=0x7fa2880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 0x7fa2880769c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa298008040 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 msgr2=0x7fa2a80cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c390 tx=0x7fa29400c850 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 shutdown_connections 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fa2880744b0 0x7fa2880769c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2a815e2f0 0x7fa2a80cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.686+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2a806fe20 0x7fa2a80cb1e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.687+0000 7fa2ace00640 1 --2- 172.21.15.123:0/2076817917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2a806d580 0x7fa2a8144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.687+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 >> 172.21.15.123:0/2076817917 conn(0x7fa2a8073200 msgr2=0x7fa2a815fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.687+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 shutdown_connections 2024-04-25T19:08:39.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.687+0000 7fa2ace00640 1 -- 172.21.15.123:0/2076817917 wait complete. 2024-04-25T19:08:39.711 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T19:08:39.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:39.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.976+0000 7efd11e00640 1 Processor -- start 2024-04-25T19:08:39.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.976+0000 7efd11e00640 1 -- start start 2024-04-25T19:08:39.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.978+0000 7efd11e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efd0c06dc90 0x7efd0c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.978+0000 7efd11e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.978+0000 7efd11e00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0c0cdf60 con 0x7efd0c147200 2024-04-25T19:08:39.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.978+0000 7efd11e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efd0c0b5100 con 0x7efd0c06dc90 2024-04-25T19:08:39.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.978+0000 7efd11e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efd0c0b4810 con 0x7efd0c1441b0 2024-04-25T19:08:39.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.979+0000 7efd11400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.979+0000 7efd10a00640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efd0c147200 0x7efd0c06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.123:54604/0 (socket says 172.21.15.123:54604) 2024-04-25T19:08:39.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.979+0000 7efd11400640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.123:48400/0 (socket says 172.21.15.123:48400) 2024-04-25T19:08:39.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.979+0000 7efd10a00640 1 -- 172.21.15.123:0/1006165549 learned_addr learned my addr 172.21.15.123:0/1006165549 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:39.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.979+0000 7efd0be00640 1 --2- 172.21.15.123:0/1006165549 >> v2:172.21.15.146:3300/0 conn(0x7efd0c06dc90 0x7efd0c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.980+0000 7efd11400640 1 -- 172.21.15.123:0/1006165549 >> v2:172.21.15.146:3300/0 conn(0x7efd0c06dc90 msgr2=0x7efd0c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:39.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.980+0000 7efd11400640 1 --2- 172.21.15.123:0/1006165549 >> v2:172.21.15.146:3300/0 conn(0x7efd0c06dc90 0x7efd0c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.980+0000 7efd11400640 1 -- 172.21.15.123:0/1006165549 >> v1:172.21.15.146:6789/0 conn(0x7efd0c147200 legacy=0x7efd0c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.980+0000 7efd11400640 1 -- 172.21.15.123:0/1006165549 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c0b4ab0 con 0x7efd0c1441b0 2024-04-25T19:08:39.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.981+0000 7efd11400640 1 --2- 172.21.15.123:0/1006165549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd04009d90 tx=0x7efd0402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e3aa4495f4002b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.985+0000 7efd0b400640 1 -- 172.21.15.123:0/1006165549 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04031600 con 0x7efd0c1441b0 2024-04-25T19:08:39.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.985+0000 7efd0b400640 1 -- 172.21.15.123:0/1006165549 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd040317a0 con 0x7efd0c1441b0 2024-04-25T19:08:39.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.985+0000 7efd0b400640 1 -- 172.21.15.123:0/1006165549 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04031b70 con 0x7efd0c1441b0 2024-04-25T19:08:39.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.985+0000 7efd11e00640 1 -- 172.21.15.123:0/1006165549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 msgr2=0x7efd0c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.985+0000 7efd11e00640 1 --2- 172.21.15.123:0/1006165549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd04009d90 tx=0x7efd0402a5d0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 -- 172.21.15.123:0/1006165549 shutdown_connections 2024-04-25T19:08:39.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 --2- 172.21.15.123:0/1006165549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c1441b0 0x7efd0c06f150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 --2- 172.21.15.123:0/1006165549 >> v2:172.21.15.146:3300/0 conn(0x7efd0c06dc90 0x7efd0c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 -- 172.21.15.123:0/1006165549 >> 172.21.15.123:0/1006165549 conn(0x7efd0c072fd0 msgr2=0x7efd0c0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:39.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 -- 172.21.15.123:0/1006165549 shutdown_connections 2024-04-25T19:08:39.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.986+0000 7efd11e00640 1 -- 172.21.15.123:0/1006165549 wait complete. 2024-04-25T19:08:39.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.988+0000 7efd11e00640 1 Processor -- start 2024-04-25T19:08:39.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.988+0000 7efd11e00640 1 -- start start 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0c1441b0 0x7efd0c150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 0x7efd0c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c13e460 0x7efd0c145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c0b5590 con 0x7efd0c147200 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c147a10 con 0x7efd0c1441b0 2024-04-25T19:08:39.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd11e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efd0c1445c0 con 0x7efd0c13e460 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 0x7efd0c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 0x7efd0c13df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48154/0 (socket says 172.21.15.123:48154) 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 -- 172.21.15.123:0/2267014494 learned_addr learned my addr 172.21.15.123:0/2267014494 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 -- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c13e460 msgr2=0x7efd0c145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd0c13e460 0x7efd0c145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 -- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0c1441b0 msgr2=0x7efd0c150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0c1441b0 0x7efd0c150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:39.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.990+0000 7efd0be00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c150890 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd0be00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 0x7efd0c13df20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e8a0 tx=0x7efcf800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8017070 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf800c940 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c13ec70 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800cd30 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.991+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c13f230 con 0x7efd0c147200 2024-04-25T19:08:39.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.992+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd0c0b6380 con 0x7efd0c147200 2024-04-25T19:08:39.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.993+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcf801a070 con 0x7efd0c147200 2024-04-25T19:08:39.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.994+0000 7efd08c00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcec0744b0 0x7efcec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.994+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7efcf80953f0 con 0x7efd0c147200 2024-04-25T19:08:39.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd10a00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcec0744b0 0x7efcec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd08c00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7efcec07a9a0 0x7efcec07ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:39.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcf8016ec0 con 0x7efcec07a9a0 2024-04-25T19:08:39.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efcf80957a0 con 0x7efd0c147200 2024-04-25T19:08:39.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd10a00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcec0744b0 0x7efcec0769c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efd00000f60 tx=0x7efd00007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:39.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.995+0000 7efd11400640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7efcec07a9a0 0x7efcec07ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:39.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.996+0000 7efd11400640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7efcec07a9a0 0x7efcec07ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:39.999+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== osd.4 v2:172.21.15.146:6801/1889014083 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcf8016ec0 con 0x7efcec07a9a0 2024-04-25T19:08:40.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.012+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 --> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd0c050740 con 0x7efcec07a9a0 2024-04-25T19:08:40.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.013+0000 7efd08c00640 1 -- 172.21.15.123:0/2267014494 <== osd.4 v2:172.21.15.146:6801/1889014083 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd0c050740 con 0x7efcec07a9a0 2024-04-25T19:08:40.015 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:40.015 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:40.016 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:40.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7efcec07a9a0 msgr2=0x7efcec07ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.146:6801/1889014083,v1:172.21.15.146:6803/1889014083] conn(0x7efcec07a9a0 0x7efcec07ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcec0744b0 msgr2=0x7efcec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7efcec0744b0 0x7efcec0769c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efd00000f60 tx=0x7efd00007a80 comp rx=0 tx=0).stop 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 msgr2=0x7efd0c13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 --2- 172.21.15.123:0/2267014494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0c147200 0x7efd0c13df20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e8a0 tx=0x7efcf800ed60 comp rx=0 tx=0).stop 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11400640 1 -- 172.21.15.123:0/2267014494 reap_dead start 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 shutdown_connections 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 >> 172.21.15.123:0/2267014494 conn(0x7efd0c072fd0 msgr2=0x7efd0c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 shutdown_connections 2024-04-25T19:08:40.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.014+0000 7efd11e00640 1 -- 172.21.15.123:0/2267014494 wait complete. 2024-04-25T19:08:40.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T19:08:40.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.302+0000 7f4797e00640 1 Processor -- start 2024-04-25T19:08:40.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.302+0000 7f4797e00640 1 -- start start 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.304+0000 7f4797e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f479806dc90 0x7f4798142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.304+0000 7f4797e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4798142a70 0x7f47980b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.304+0000 7f4797e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47980b5c70 con 0x7f479813fbf0 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4797e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f47980b5f50 con 0x7f4798142a70 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4797e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47980b6230 con 0x7f479806dc90 2024-04-25T19:08:40.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4796a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f479813fbf0 0x7f479806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56464/0 (socket says 172.21.15.123:56464) 2024-04-25T19:08:40.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4796a00640 1 -- 172.21.15.123:0/3178015758 learned_addr learned my addr 172.21.15.123:0/3178015758 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:40.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4797400640 1 --2- 172.21.15.123:0/3178015758 >> v2:172.21.15.123:3300/0 conn(0x7f4798142a70 0x7f47980b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f478fe00640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f479806dc90 0x7f4798142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4796000640 1 -- 172.21.15.123:0/3178015758 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271036485 0 0) 0x7f47980b5c70 con 0x7f479813fbf0 2024-04-25T19:08:40.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.305+0000 7f4796000640 1 -- 172.21.15.123:0/3178015758 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f477c005730 con 0x7f479813fbf0 2024-04-25T19:08:40.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.306+0000 7f4797400640 1 -- 172.21.15.123:0/3178015758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f479806dc90 msgr2=0x7f4798142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:40.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.306+0000 7f4797400640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f479806dc90 0x7f4798142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.306+0000 7f4797400640 1 -- 172.21.15.123:0/3178015758 >> v1:172.21.15.123:6789/0 conn(0x7f479813fbf0 legacy=0x7f479806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.306+0000 7f4797400640 1 -- 172.21.15.123:0/3178015758 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47980b6510 con 0x7f4798142a70 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.307+0000 7f4797400640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4798142a70 0x7f47980b4640 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4790009d90 tx=0x7f479002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d0ba1fff94e4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.307+0000 7f4796000640 1 -- 172.21.15.123:0/3178015758 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790007740 con 0x7f4798142a70 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.307+0000 7f4796000640 1 -- 172.21.15.123:0/3178015758 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47900078e0 con 0x7f4798142a70 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.307+0000 7f4796000640 1 -- 172.21.15.123:0/3178015758 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790007cb0 con 0x7f4798142a70 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 -- 172.21.15.123:0/3178015758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4798142a70 msgr2=0x7f47980b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4798142a70 0x7f47980b4640 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4790009d90 tx=0x7f479002a660 comp rx=0 tx=0).stop 2024-04-25T19:08:40.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 -- 172.21.15.123:0/3178015758 shutdown_connections 2024-04-25T19:08:40.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4798142a70 0x7f47980b4640 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 --2- 172.21.15.123:0/3178015758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f479806dc90 0x7f4798142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 -- 172.21.15.123:0/3178015758 >> 172.21.15.123:0/3178015758 conn(0x7f4798073200 msgr2=0x7f4798073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:40.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 -- 172.21.15.123:0/3178015758 shutdown_connections 2024-04-25T19:08:40.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.308+0000 7f4797e00640 1 -- 172.21.15.123:0/3178015758 wait complete. 2024-04-25T19:08:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.310+0000 7f4797e00640 1 Processor -- start 2024-04-25T19:08:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.310+0000 7f4797e00640 1 -- start start 2024-04-25T19:08:40.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4798142a70 0x7f47981565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47981f9670 0x7f4798156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4798143000 con 0x7f479813fbf0 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4798142e80 con 0x7f47981f9670 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4797e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47980b4b80 con 0x7f4798142a70 2024-04-25T19:08:40.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48182/0 (socket says 172.21.15.123:48182) 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 -- 172.21.15.123:0/399449652 learned_addr learned my addr 172.21.15.123:0/399449652 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 -- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4798142a70 msgr2=0x7f47981565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4798142a70 0x7f47981565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 -- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47981f9670 msgr2=0x7f4798156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47981f9670 0x7f4798156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 -- 172.21.15.123:0/399449652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4798140e00 con 0x7f479813fbf0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.312+0000 7f4796a00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f478800c3c0 tx=0x7f478800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.313+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788017070 con 0x7f479813fbf0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.313+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788005ce0 con 0x7f479813fbf0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.313+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880072f0 con 0x7f479813fbf0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.313+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47981412e0 con 0x7f479813fbf0 2024-04-25T19:08:40.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.313+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981418a0 con 0x7f479813fbf0 2024-04-25T19:08:40.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.314+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47980d0c90 con 0x7f479813fbf0 2024-04-25T19:08:40.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.315+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f478801a070 con 0x7f479813fbf0 2024-04-25T19:08:40.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.316+0000 7f478ea00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 0x7f47740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.316+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f47880944d0 con 0x7f479813fbf0 2024-04-25T19:08:40.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.317+0000 7f478fe00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 0x7f47740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.317+0000 7f478fe00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 0x7f47740769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.321+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47980d0c90 con 0x7f479813fbf0 2024-04-25T19:08:40.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.519+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f4798050740 con 0x7f47740744b0 2024-04-25T19:08:40.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.520+0000 7f478ea00640 1 -- 172.21.15.123:0/399449652 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4798050740 con 0x7f47740744b0 2024-04-25T19:08:40.522 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T19:08:40.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 msgr2=0x7f47740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 0x7f47740769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780008040 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 msgr2=0x7f4798154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f478800c3c0 tx=0x7f478800c880 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 shutdown_connections 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f47740744b0 0x7f47740769c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47981f9670 0x7f4798156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4798142a70 0x7f47981565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 --2- 172.21.15.123:0/399449652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f479813fbf0 0x7f4798154240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.523+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 >> 172.21.15.123:0/399449652 conn(0x7f4798073200 msgr2=0x7f479806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.524+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 shutdown_connections 2024-04-25T19:08:40.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.524+0000 7f4797e00640 1 -- 172.21.15.123:0/399449652 wait complete. 2024-04-25T19:08:40.554 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T19:08:40.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:40.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.826+0000 7f3806a00640 1 Processor -- start 2024-04-25T19:08:40.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.826+0000 7f3806a00640 1 -- start start 2024-04-25T19:08:40.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.828+0000 7f3806a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f380006d580 0x7f380006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.828+0000 7f3806a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380006ded0 con 0x7f3800160820 2024-04-25T19:08:40.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.828+0000 7f3806a00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3800163fc0 con 0x7f3800156f00 2024-04-25T19:08:40.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.828+0000 7f3806a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f38000cf2a0 con 0x7f380006d580 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3804c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f380006d580 0x7f380006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3806000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3800160820 0x7f3800160c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56468/0 (socket says 172.21.15.123:56468) 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3806000640 1 -- 172.21.15.123:0/721023122 learned_addr learned my addr 172.21.15.123:0/721023122 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3804c00640 1 -- 172.21.15.123:0/721023122 >> v1:172.21.15.146:6789/0 conn(0x7f3800156f00 legacy=0x7f3800070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f37ffe00640 1 -- 172.21.15.123:0/721023122 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210406105 0 0) 0x7f37f8001490 con 0x7f3800160820 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3804c00640 1 -- 172.21.15.123:0/721023122 >> v1:172.21.15.123:6789/0 conn(0x7f3800160820 legacy=0x7f3800160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.829+0000 7f3804c00640 1 -- 172.21.15.123:0/721023122 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38001642a0 con 0x7f380006d580 2024-04-25T19:08:40.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.831+0000 7f3804c00640 1 --2- 172.21.15.123:0/721023122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f380006d580 0x7f380006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097a0 tx=0x7f37f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591057c71bba9db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.831+0000 7f37ffe00640 1 -- 172.21.15.123:0/721023122 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4037070 con 0x7f380006d580 2024-04-25T19:08:40.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.831+0000 7f37ffe00640 1 -- 172.21.15.123:0/721023122 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f4032440 con 0x7f380006d580 2024-04-25T19:08:40.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.831+0000 7f37ffe00640 1 -- 172.21.15.123:0/721023122 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4032810 con 0x7f380006d580 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 -- 172.21.15.123:0/721023122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f380006d580 msgr2=0x7f380006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 --2- 172.21.15.123:0/721023122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f380006d580 0x7f380006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097a0 tx=0x7f37f402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 -- 172.21.15.123:0/721023122 shutdown_connections 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 --2- 172.21.15.123:0/721023122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f380006d580 0x7f380006d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 -- 172.21.15.123:0/721023122 >> 172.21.15.123:0/721023122 conn(0x7f3800072fd0 msgr2=0x7f38000733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 -- 172.21.15.123:0/721023122 shutdown_connections 2024-04-25T19:08:40.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.832+0000 7f3806a00640 1 -- 172.21.15.123:0/721023122 wait complete. 2024-04-25T19:08:40.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.834+0000 7f3806a00640 1 Processor -- start 2024-04-25T19:08:40.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.835+0000 7f3806a00640 1 -- start start 2024-04-25T19:08:40.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f380006d580 0x7f38001f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 0x7f38000b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3800160820 0x7f3800147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3800157310 con 0x7f3800156f00 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3800164860 con 0x7f380006d580 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3806a00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3800157490 con 0x7f3800160820 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3804c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 0x7f38000b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3804c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 0x7f38000b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48194/0 (socket says 172.21.15.123:48194) 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.836+0000 7f3804c00640 1 -- 172.21.15.123:0/1402894014 learned_addr learned my addr 172.21.15.123:0/1402894014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3805600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f380006d580 0x7f38001f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 -- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3800160820 msgr2=0x7f3800147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3800160820 0x7f3800147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 -- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f380006d580 msgr2=0x7f38001f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f380006d580 0x7f38001f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38000b5950 con 0x7f3800156f00 2024-04-25T19:08:40.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3805600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f380006d580 0x7f38001f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:40.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3804c00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 0x7f38000b5410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f37f4000c00 tx=0x7f37f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4037040 con 0x7f3800156f00 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f4042070 con 0x7f3800156f00 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4030a70 con 0x7f3800156f00 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.837+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38000b4650 con 0x7f3800156f00 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.838+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38000b4c10 con 0x7f3800156f00 2024-04-25T19:08:40.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.838+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3800050740 con 0x7f3800156f00 2024-04-25T19:08:40.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.840+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f37f4030c10 con 0x7f3800156f00 2024-04-25T19:08:40.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.841+0000 7f37fd600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f37e40744b0 0x7f37e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.841+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f37f40b68e0 con 0x7f3800156f00 2024-04-25T19:08:40.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f37fd600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f37e407a9a0 0x7f37e407ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37f403ee50 con 0x7f37e407a9a0 2024-04-25T19:08:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f37f40b6c90 con 0x7f3800156f00 2024-04-25T19:08:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f3805600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f37e40744b0 0x7f37e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f3806000640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f37e407a9a0 0x7f37e407ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.842+0000 7f3805600640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f37e40744b0 0x7f37e40769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f37f00015e0 tx=0x7f37f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.843+0000 7f3806000640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f37e407a9a0 0x7f37e407ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:40.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.846+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== osd.5 v2:172.21.15.146:6800/2028526652 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37f403ee50 con 0x7f37e407a9a0 2024-04-25T19:08:40.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.860+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 --> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f380015a2d0 con 0x7f37e407a9a0 2024-04-25T19:08:40.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.861+0000 7f37fd600640 1 -- 172.21.15.123:0/1402894014 <== osd.5 v2:172.21.15.146:6800/2028526652 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f380015a2d0 con 0x7f37e407a9a0 2024-04-25T19:08:40.863 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:40.863 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:40.863 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:40.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f37e407a9a0 msgr2=0x7f37e407ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.146:6800/2028526652,v1:172.21.15.146:6802/2028526652] conn(0x7f37e407a9a0 0x7f37e407ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f37e40744b0 msgr2=0x7f37e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f37e40744b0 0x7f37e40769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f37f00015e0 tx=0x7f37f0009040 comp rx=0 tx=0).stop 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 msgr2=0x7f38000b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 --2- 172.21.15.123:0/1402894014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3800156f00 0x7f38000b5410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f37f4000c00 tx=0x7f37f40046c0 comp rx=0 tx=0).stop 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806000640 1 -- 172.21.15.123:0/1402894014 reap_dead start 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 shutdown_connections 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 >> 172.21.15.123:0/1402894014 conn(0x7f3800072fd0 msgr2=0x7f38000700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 shutdown_connections 2024-04-25T19:08:40.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:40.862+0000 7f3806a00640 1 -- 172.21.15.123:0/1402894014 wait complete. 2024-04-25T19:08:40.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T19:08:41.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.152+0000 7fbda4000640 1 Processor -- start 2024-04-25T19:08:41.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.152+0000 7fbda4000640 1 -- start start 2024-04-25T19:08:41.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda4000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd9c0d1120 0x7fbd9c070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda4000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c154950 0x7fbd9c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda4000640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c0aec30 con 0x7fbd9c161b10 2024-04-25T19:08:41.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda4000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c0d1530 con 0x7fbd9c0d1120 2024-04-25T19:08:41.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda4000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0aef10 con 0x7fbd9c154950 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd9c0d1120 0x7fbd9c070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.154+0000 7fbda1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd9c0d1120 0x7fbd9c070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48208/0 (socket says 172.21.15.123:48208) 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.155+0000 7fbda1600640 1 -- 172.21.15.123:0/662422082 learned_addr learned my addr 172.21.15.123:0/662422082 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.155+0000 7fbda1600640 1 -- 172.21.15.123:0/662422082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c154950 msgr2=0x7fbd9c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.155+0000 7fbda1600640 1 --2- 172.21.15.123:0/662422082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c154950 0x7fbd9c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.155+0000 7fbda1600640 1 -- 172.21.15.123:0/662422082 >> v1:172.21.15.146:6789/0 conn(0x7fbd9c161b10 legacy=0x7fbd9c161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.155+0000 7fbda1600640 1 -- 172.21.15.123:0/662422082 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0af1f0 con 0x7fbd9c0d1120 2024-04-25T19:08:41.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.156+0000 7fbda1600640 1 --2- 172.21.15.123:0/662422082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c070ef0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009870 tx=0x7fbd9002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431fcaba2bef1337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.156+0000 7fbd9be00640 1 -- 172.21.15.123:0/662422082 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90037070 con 0x7fbd9c0d1120 2024-04-25T19:08:41.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.156+0000 7fbd9be00640 1 -- 172.21.15.123:0/662422082 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9002ae20 con 0x7fbd9c0d1120 2024-04-25T19:08:41.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbd9be00640 1 -- 172.21.15.123:0/662422082 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90032660 con 0x7fbd9c0d1120 2024-04-25T19:08:41.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 -- 172.21.15.123:0/662422082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 msgr2=0x7fbd9c070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 --2- 172.21.15.123:0/662422082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c070ef0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009870 tx=0x7fbd9002a110 comp rx=0 tx=0).stop 2024-04-25T19:08:41.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 -- 172.21.15.123:0/662422082 shutdown_connections 2024-04-25T19:08:41.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 --2- 172.21.15.123:0/662422082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c154950 0x7fbd9c154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 --2- 172.21.15.123:0/662422082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c070ef0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 -- 172.21.15.123:0/662422082 >> 172.21.15.123:0/662422082 conn(0x7fbd9c0d1ae0 msgr2=0x7fbd9c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:41.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.157+0000 7fbda4000640 1 -- 172.21.15.123:0/662422082 shutdown_connections 2024-04-25T19:08:41.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.158+0000 7fbda4000640 1 -- 172.21.15.123:0/662422082 wait complete. 2024-04-25T19:08:41.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.160+0000 7fbda4000640 1 Processor -- start 2024-04-25T19:08:41.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.160+0000 7fbda4000640 1 -- start start 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 0x7fbd9c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 0x7fbd9c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1552a0 con 0x7fbd9c0d1120 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c071430 con 0x7fbd9c154950 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda4000640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c150b80 con 0x7fbd9c161b10 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48212/0 (socket says 172.21.15.123:48212) 2024-04-25T19:08:41.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 -- 172.21.15.123:0/2973121921 learned_addr learned my addr 172.21.15.123:0/2973121921 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda2000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 0x7fbd9c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 -- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 msgr2=0x7fbd9c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 0x7fbd9c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 -- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 msgr2=0x7fbd9c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda0c00640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 0x7fbd9c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 0x7fbd9c14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda1600640 1 -- 172.21.15.123:0/2973121921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c1e7ab0 con 0x7fbd9c0d1120 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.162+0000 7fbda2000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 0x7fbd9c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbda0c00640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 0x7fbd9c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbda1600640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009870 tx=0x7fbd900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90042070 con 0x7fbd9c0d1120 2024-04-25T19:08:41.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9003e660 con 0x7fbd9c0d1120 2024-04-25T19:08:41.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90037040 con 0x7fbd9c0d1120 2024-04-25T19:08:41.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c1e7d90 con 0x7fbd9c0d1120 2024-04-25T19:08:41.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.163+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c1508c0 con 0x7fbd9c0d1120 2024-04-25T19:08:41.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.165+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c076a40 con 0x7fbd9c0d1120 2024-04-25T19:08:41.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.166+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd90045070 con 0x7fbd9c0d1120 2024-04-25T19:08:41.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.167+0000 7fbd99600640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 0x7fbd7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.167+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fbd900b59e0 con 0x7fbd9c0d1120 2024-04-25T19:08:41.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.167+0000 7fbda0c00640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 0x7fbd7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.168+0000 7fbda0c00640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 0x7fbd7c0769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1e95a0 tx=0x7fbd88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.172+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd90080360 con 0x7fbd9c0d1120 2024-04-25T19:08:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.372+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fbd9c070760 con 0x7fbd7c0744b0 2024-04-25T19:08:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.373+0000 7fbd99600640 1 -- 172.21.15.123:0/2973121921 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbd9c070760 con 0x7fbd7c0744b0 2024-04-25T19:08:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T19:08:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 msgr2=0x7fbd7c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 0x7fbd7c0769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1e95a0 tx=0x7fbd88008040 comp rx=0 tx=0).stop 2024-04-25T19:08:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 msgr2=0x7fbd9c160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009870 tx=0x7fbd900046c0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 shutdown_connections 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fbd7c0744b0 0x7fbd7c0769c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd9c161b10 0x7fbd9c0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd9c154950 0x7fbd9c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 --2- 172.21.15.123:0/2973121921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d1120 0x7fbd9c160f90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 >> 172.21.15.123:0/2973121921 conn(0x7fbd9c0d1ae0 msgr2=0x7fbd9c15e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 shutdown_connections 2024-04-25T19:08:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.376+0000 7fbda4000640 1 -- 172.21.15.123:0/2973121921 wait complete. 2024-04-25T19:08:41.402 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T19:08:41.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:41.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.679+0000 7f6595800640 1 Processor -- start 2024-04-25T19:08:41.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.679+0000 7f6595800640 1 -- start start 2024-04-25T19:08:41.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.681+0000 7f6595800640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f65900cd8d0 0x7f65901441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.681+0000 7f6595800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900cdce0 con 0x7f6590147200 2024-04-25T19:08:41.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.681+0000 7f6595800640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900cdf60 con 0x7f659006dc90 2024-04-25T19:08:41.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f6595800640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f65900b5100 con 0x7f65900cd8d0 2024-04-25T19:08:41.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f658fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6590147200 0x7f659006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56470/0 (socket says 172.21.15.123:56470) 2024-04-25T19:08:41.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f658fe00640 1 -- 172.21.15.123:0/2282388692 learned_addr learned my addr 172.21.15.123:0/2282388692 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475421183 0 0) 0x7f6580001110 con 0x7f6590147200 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6574003520 con 0x7f6590147200 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.682+0000 7f6594e00640 1 --2- 172.21.15.123:0/2282388692 >> v2:172.21.15.146:3300/0 conn(0x7f65900cd8d0 0x7f65901441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2374655252 0 0) 0x7f6574003520 con 0x7f6590147200 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 >> v2:172.21.15.146:3300/0 conn(0x7f65900cd8d0 msgr2=0x7f65901441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 --2- 172.21.15.123:0/2282388692 >> v2:172.21.15.146:3300/0 conn(0x7f65900cd8d0 0x7f65901441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 >> v1:172.21.15.146:6789/0 conn(0x7f659006dc90 legacy=0x7f659006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65900b4810 con 0x7f6590147200 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7f6580001840 con 0x7f6590147200 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 >> v1:172.21.15.123:6789/0 conn(0x7f6590147200 legacy=0x7f659006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 0x7f6574003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6574004800 0x7f6574004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6574005300 0x7f657400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65900b5100 con 0x7f65740039b0 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f657400c2a0 con 0x7f6574004800 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f657400c4a0 con 0x7f6574005300 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.683+0000 7f658fe00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 0x7f6574003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658f400640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6574004800 0x7f6574004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f6594e00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6574005300 0x7f657400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658fe00640 1 -- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6574005300 msgr2=0x7f657400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658fe00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6574005300 0x7f657400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658fe00640 1 -- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6574004800 msgr2=0x7f6574004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658fe00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6574004800 0x7f6574004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658fe00640 1 -- 172.21.15.123:0/2282388692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6574003520 con 0x7f65740039b0 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f6594e00640 1 -- 172.21.15.123:0/2282388692 reap_dead start 2024-04-25T19:08:41.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.684+0000 7f658f400640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6574004800 0x7f6574004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.685+0000 7f658fe00640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 0x7f6574003e20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f658000c4c0 tx=0x7f6574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9299f0072710ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.685+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658000c0e0 con 0x7f65740039b0 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.685+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580034070 con 0x7f65740039b0 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.685+0000 7f658ea00640 1 -- 172.21.15.123:0/2282388692 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580005d70 con 0x7f65740039b0 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 -- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 msgr2=0x7f6574003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 0x7f6574003e20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f658000c4c0 tx=0x7f6574005830 comp rx=0 tx=0).stop 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 -- 172.21.15.123:0/2282388692 shutdown_connections 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 --2- 172.21.15.123:0/2282388692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65740039b0 0x7f6574003e20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 -- 172.21.15.123:0/2282388692 >> 172.21.15.123:0/2282388692 conn(0x7f6590072fd0 msgr2=0x7f65900733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:41.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 -- 172.21.15.123:0/2282388692 shutdown_connections 2024-04-25T19:08:41.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.686+0000 7f6595800640 1 -- 172.21.15.123:0/2282388692 wait complete. 2024-04-25T19:08:41.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.688+0000 7f6595800640 1 Processor -- start 2024-04-25T19:08:41.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.689+0000 7f6595800640 1 -- start start 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65740039b0 0x7f65901f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 0x7f6590070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65901621c0 0x7f6590070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6590147a10 con 0x7f6590161db0 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f65900b4d90 con 0x7f65901621c0 2024-04-25T19:08:41.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f6595800640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f65900b5590 con 0x7f65740039b0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f658f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 0x7f6590070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f658f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 0x7f6590070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48228/0 (socket says 172.21.15.123:48228) 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.690+0000 7f658f400640 1 -- 172.21.15.123:0/3182033502 learned_addr learned my addr 172.21.15.123:0/3182033502 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658fe00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65740039b0 0x7f65901f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6594e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65901621c0 0x7f6590070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 -- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65740039b0 msgr2=0x7f65901f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65740039b0 0x7f65901f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 -- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65901621c0 msgr2=0x7f6590070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65901621c0 0x7f6590070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590151320 con 0x7f6590161db0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6594e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65901621c0 0x7f6590070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f658f400640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 0x7f6590070560 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d400 tx=0x7f657c00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c018070 con 0x7f6590161db0 2024-04-25T19:08:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c003610 con 0x7f6590161db0 2024-04-25T19:08:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c003a40 con 0x7f6590161db0 2024-04-25T19:08:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.691+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6590151600 con 0x7f6590161db0 2024-04-25T19:08:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.692+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6590151bc0 con 0x7f6590161db0 2024-04-25T19:08:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.693+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65900718e0 con 0x7f6590161db0 2024-04-25T19:08:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.694+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f657c01b070 con 0x7f6590161db0 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f6587e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65700744b0 0x7f65700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f657c0948b0 con 0x7f6590161db0 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f6587e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7f657007a9a0 0x7f657007ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f658fe00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65700744b0 0x7f65700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f657c01ce50 con 0x7f657007a9a0 2024-04-25T19:08:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.695+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f657c0992a0 con 0x7f6590161db0 2024-04-25T19:08:41.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.696+0000 7f658fe00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65700744b0 0x7f65700769c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f658000c490 tx=0x7f658002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.696+0000 7f6594e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7f657007a9a0 0x7f657007ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:41.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.697+0000 7f6594e00640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7f657007a9a0 0x7f657007ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.699+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== osd.6 v2:172.21.15.146:6824/510483223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f657c01ce50 con 0x7f657007a9a0 2024-04-25T19:08:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.714+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 --> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f65901510d0 con 0x7f657007a9a0 2024-04-25T19:08:41.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.715+0000 7f6587e00640 1 -- 172.21.15.123:0/3182033502 <== osd.6 v2:172.21.15.146:6824/510483223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f65901510d0 con 0x7f657007a9a0 2024-04-25T19:08:41.717 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:41.717 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:41.717 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.715+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7f657007a9a0 msgr2=0x7f657007ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.715+0000 7f6595800640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.146:6824/510483223,v1:172.21.15.146:6825/510483223] conn(0x7f657007a9a0 0x7f657007ce30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65700744b0 msgr2=0x7f65700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f65700744b0 0x7f65700769c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f658000c490 tx=0x7f658002e000 comp rx=0 tx=0).stop 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 msgr2=0x7f6590070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:41.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 --2- 172.21.15.123:0/3182033502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6590161db0 0x7f6590070560 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d400 tx=0x7f657c00d8c0 comp rx=0 tx=0).stop 2024-04-25T19:08:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6594e00640 1 -- 172.21.15.123:0/3182033502 reap_dead start 2024-04-25T19:08:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 shutdown_connections 2024-04-25T19:08:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 >> 172.21.15.123:0/3182033502 conn(0x7f6590072fd0 msgr2=0x7f65900b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 shutdown_connections 2024-04-25T19:08:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:41.716+0000 7f6595800640 1 -- 172.21.15.123:0/3182033502 wait complete. 2024-04-25T19:08:41.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T19:08:42.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.010+0000 7fabafc00640 1 Processor -- start 2024-04-25T19:08:42.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.010+0000 7fabafc00640 1 -- start start 2024-04-25T19:08:42.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.012+0000 7fabafc00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba806cc80 0x7faba806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabafc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba80afe00 con 0x7faba8156cd0 2024-04-25T19:08:42.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabafc00640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba80b00a0 con 0x7faba815d7a0 2024-04-25T19:08:42.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabafc00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faba815dbb0 con 0x7faba806cc80 2024-04-25T19:08:42.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabad200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7faba8156cd0 0x7faba8070e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56472/0 (socket says 172.21.15.123:56472) 2024-04-25T19:08:42.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabad200640 1 -- 172.21.15.123:0/3082756275 learned_addr learned my addr 172.21.15.123:0/3082756275 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435553756 0 0) 0x7faba80afe00 con 0x7faba8156cd0 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab8c005730 con 0x7faba8156cd0 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.013+0000 7fabadc00640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba806cc80 0x7faba806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3008670586 0 0) 0x7fab8c005730 con 0x7faba8156cd0 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba806cc80 msgr2=0x7faba806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba806cc80 0x7faba806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 >> v1:172.21.15.146:6789/0 conn(0x7faba815d7a0 legacy=0x7faba806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba80b0380 con 0x7faba8156cd0 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fab980019f0 con 0x7faba8156cd0 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 >> v1:172.21.15.123:6789/0 conn(0x7faba8156cd0 legacy=0x7faba8070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab8c005bf0 0x7fab8c006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c006a10 0x7fab8c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8c007560 0x7fab8c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faba815dbb0 con 0x7fab8c005bf0 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00e3c0 con 0x7fab8c006a10 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c00e5c0 con 0x7fab8c007560 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7fabadc00640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba806cc80 0x7faba806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.014+0000 7fabad200640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab8c005bf0 0x7fab8c006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabad200640 1 -- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8c007560 msgr2=0x7fab8c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabadc00640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8c007560 0x7fab8c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabad200640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab8c007560 0x7fab8c00dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7faba7e00640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c006a10 0x7fab8c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabad200640 1 -- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c006a10 msgr2=0x7fab8c006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabad200640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c006a10 0x7fab8c006e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.015+0000 7fabad200640 1 -- 172.21.15.123:0/3082756275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c005730 con 0x7fab8c005bf0 2024-04-25T19:08:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.016+0000 7fabad200640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab8c005bf0 0x7fab8c006060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fab9800a760 tx=0x7fab8c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d6fb9a68e7e2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.016+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800a3b0 con 0x7fab8c005bf0 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.016+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9802f070 con 0x7fab8c005bf0 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.016+0000 7faba7400640 1 -- 172.21.15.123:0/3082756275 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9802bbe0 con 0x7fab8c005bf0 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 -- 172.21.15.123:0/3082756275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab8c005bf0 msgr2=0x7fab8c006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 --2- 172.21.15.123:0/3082756275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab8c005bf0 0x7fab8c006060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fab9800a760 tx=0x7fab8c007a40 comp rx=0 tx=0).stop 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabadc00640 1 -- 172.21.15.123:0/3082756275 reap_dead start 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 -- 172.21.15.123:0/3082756275 shutdown_connections 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 -- 172.21.15.123:0/3082756275 >> 172.21.15.123:0/3082756275 conn(0x7faba80d1ae0 msgr2=0x7faba80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:42.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 -- 172.21.15.123:0/3082756275 shutdown_connections 2024-04-25T19:08:42.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.017+0000 7fabafc00640 1 -- 172.21.15.123:0/3082756275 wait complete. 2024-04-25T19:08:42.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.019+0000 7fabafc00640 1 Processor -- start 2024-04-25T19:08:42.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.019+0000 7fabafc00640 1 -- start start 2024-04-25T19:08:42.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c005bf0 0x7faba809be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 0x7faba809ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faba80b0e60 con 0x7faba8145fa0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faba80b0940 con 0x7fab8c005bf0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7fabafc00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faba806d090 con 0x7faba809eaf0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7faba7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7faba7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48248/0 (socket says 172.21.15.123:48248) 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.021+0000 7faba7e00640 1 -- 172.21.15.123:0/2476734751 learned_addr learned my addr 172.21.15.123:0/2476734751 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7fabad200640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c005bf0 0x7faba809be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7fabadc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 0x7faba809ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 -- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 msgr2=0x7faba809ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 0x7faba809ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 -- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c005bf0 msgr2=0x7faba809be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c005bf0 0x7faba809be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 -- 172.21.15.123:0/2476734751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba809c3d0 con 0x7faba8145fa0 2024-04-25T19:08:42.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7fabadc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 0x7faba809ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:42.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba7e00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d4d0 tx=0x7fab9000d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90005480 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab90017070 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba809ef00 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.022+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90003830 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.023+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba8151e70 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.024+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba8151be0 con 0x7faba8145fa0 2024-04-25T19:08:42.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.025+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab900039d0 con 0x7faba8145fa0 2024-04-25T19:08:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.026+0000 7faba4c00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.027+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fab90094ec0 con 0x7faba8145fa0 2024-04-25T19:08:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.027+0000 7fabad200640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.027+0000 7fabad200640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 0x7fab840769f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab98033000 tx=0x7fab98009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.031+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faba809ef00 con 0x7faba8145fa0 2024-04-25T19:08:42.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.235+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7faba809c5f0 con 0x7fab840744e0 2024-04-25T19:08:42.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.236+0000 7faba4c00640 1 -- 172.21.15.123:0/2476734751 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faba809c5f0 con 0x7fab840744e0 2024-04-25T19:08:42.238 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T19:08:42.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 msgr2=0x7fab840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 0x7fab840769f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab98033000 tx=0x7fab98009030 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 msgr2=0x7faba809e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d4d0 tx=0x7fab9000d990 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 shutdown_connections 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba809eaf0 0x7faba809ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faba8145fa0 0x7faba809e5b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 --2- 172.21.15.123:0/2476734751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab8c005bf0 0x7faba809be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.238+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 >> 172.21.15.123:0/2476734751 conn(0x7faba80d1ae0 msgr2=0x7faba80b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.239+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 shutdown_connections 2024-04-25T19:08:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.239+0000 7fabafc00640 1 -- 172.21.15.123:0/2476734751 wait complete. 2024-04-25T19:08:42.266 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T19:08:42.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:08:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.544+0000 7f8322600640 1 Processor -- start 2024-04-25T19:08:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.544+0000 7f8322600640 1 -- start start 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8322600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f831c070ac0 0x7f831c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8322600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c06d580 0x7f831c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8322600640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831c06ded0 con 0x7f831c0b4b00 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8322600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f831c0b6550 con 0x7f831c070ac0 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8322600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0b6810 con 0x7f831c06d580 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.546+0000 7f8321200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f831c070ac0 0x7f831c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f831c070ac0 0x7f831c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48254/0 (socket says 172.21.15.123:48254) 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 -- 172.21.15.123:0/3066076671 learned_addr learned my addr 172.21.15.123:0/3066076671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321c00640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c06d580 0x7f831c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 -- 172.21.15.123:0/3066076671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c06d580 msgr2=0x7f831c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c06d580 0x7f831c06d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 -- 172.21.15.123:0/3066076671 >> v1:172.21.15.146:6789/0 conn(0x7f831c0b4b00 legacy=0x7f831c0cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.547+0000 7f8321200640 1 -- 172.21.15.123:0/3066076671 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c15e320 con 0x7f831c070ac0 2024-04-25T19:08:42.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.548+0000 7f8321200640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c070ac0 0x7f831c15d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8308009870 tx=0x7f830802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73cb688a8f7f9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f831b400640 1 -- 172.21.15.123:0/3066076671 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308037070 con 0x7f831c070ac0 2024-04-25T19:08:42.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f831b400640 1 -- 172.21.15.123:0/3066076671 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8308032440 con 0x7f831c070ac0 2024-04-25T19:08:42.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f831b400640 1 -- 172.21.15.123:0/3066076671 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308032810 con 0x7f831c070ac0 2024-04-25T19:08:42.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 -- 172.21.15.123:0/3066076671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c070ac0 msgr2=0x7f831c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c070ac0 0x7f831c15d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8308009870 tx=0x7f830802a210 comp rx=0 tx=0).stop 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 -- 172.21.15.123:0/3066076671 shutdown_connections 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c06d580 0x7f831c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 --2- 172.21.15.123:0/3066076671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c070ac0 0x7f831c15d330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.549+0000 7f8322600640 1 -- 172.21.15.123:0/3066076671 >> 172.21.15.123:0/3066076671 conn(0x7f831c072fd0 msgr2=0x7f831c0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.550+0000 7f8322600640 1 -- 172.21.15.123:0/3066076671 shutdown_connections 2024-04-25T19:08:42.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.550+0000 7f8322600640 1 -- 172.21.15.123:0/3066076671 wait complete. 2024-04-25T19:08:42.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.552+0000 7f8322600640 1 Processor -- start 2024-04-25T19:08:42.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.552+0000 7f8322600640 1 -- start start 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 0x7f831c1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f831c070ac0 0x7f831c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c0b4b00 0x7f831c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f831c15e040 con 0x7f831c06d580 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f831c15dd60 con 0x7f831c070ac0 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.553+0000 7f8322600640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0b4f10 con 0x7f831c0b4b00 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 0x7f831c1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 0x7f831c1470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48258/0 (socket says 172.21.15.123:48258) 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 -- 172.21.15.123:0/2764145142 learned_addr learned my addr 172.21.15.123:0/2764145142 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 -- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c0b4b00 msgr2=0x7f831c14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f831c0b4b00 0x7f831c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 -- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f831c070ac0 msgr2=0x7f831c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f831c070ac0 0x7f831c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.554+0000 7f8321200640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c16a460 con 0x7f831c06d580 2024-04-25T19:08:42.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8321200640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 0x7f831c1470b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8308009870 tx=0x7f83080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308037040 con 0x7f831c06d580 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8308042070 con 0x7f831c06d580 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c147770 con 0x7f831c06d580 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308030bb0 con 0x7f831c06d580 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.555+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c159c50 con 0x7f831c06d580 2024-04-25T19:08:42.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.556+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f831c050740 con 0x7f831c06d580 2024-04-25T19:08:42.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.558+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8308047460 con 0x7f831c06d580 2024-04-25T19:08:42.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.558+0000 7f8318c00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f82fc0744e0 0x7f82fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.559+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f83080b68f0 con 0x7f831c06d580 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.559+0000 7f831be00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f82fc0744e0 0x7f82fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.559+0000 7f8318c00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7f82fc07a9d0 0x7f82fc07ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.559+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8308032440 con 0x7f82fc07a9d0 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.559+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f83080b6ca0 con 0x7f831c06d580 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.560+0000 7f831be00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f82fc0744e0 0x7f82fc0769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f831c0b3220 tx=0x7f830c009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.560+0000 7f8321c00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7f82fc07a9d0 0x7f82fc07ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.560+0000 7f8321c00640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7f82fc07a9d0 0x7f82fc07ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.563+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== osd.7 v2:172.21.15.146:6816/3816144943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8308032440 con 0x7f82fc07a9d0 2024-04-25T19:08:42.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.577+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 --> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f831c073ee0 con 0x7f82fc07a9d0 2024-04-25T19:08:42.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.578+0000 7f8318c00640 1 -- 172.21.15.123:0/2764145142 <== osd.7 v2:172.21.15.146:6816/3816144943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f831c073ee0 con 0x7f82fc07a9d0 2024-04-25T19:08:42.580 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-25T19:08:42.580 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:08:42.581 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-25T19:08:42.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7f82fc07a9d0 msgr2=0x7f82fc07ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.146:6816/3816144943,v1:172.21.15.146:6817/3816144943] conn(0x7f82fc07a9d0 0x7f82fc07ce60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f82fc0744e0 msgr2=0x7f82fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f82fc0744e0 0x7f82fc0769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f831c0b3220 tx=0x7f830c009230 comp rx=0 tx=0).stop 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 msgr2=0x7f831c1470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 --2- 172.21.15.123:0/2764145142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c06d580 0x7f831c1470b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8308009870 tx=0x7f83080046c0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8321c00640 1 -- 172.21.15.123:0/2764145142 reap_dead start 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 shutdown_connections 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 >> 172.21.15.123:0/2764145142 conn(0x7f831c072fd0 msgr2=0x7f831c1517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 shutdown_connections 2024-04-25T19:08:42.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.579+0000 7f8322600640 1 -- 172.21.15.123:0/2764145142 wait complete. 2024-04-25T19:08:42.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T19:08:42.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.877+0000 7fd223c00640 1 Processor -- start 2024-04-25T19:08:42.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.878+0000 7fd223c00640 1 -- start start 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd223c00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd21c161b10 0x7fd21c161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd223c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd21c154950 0x7fd21c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd223c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c071600 con 0x7fd21c0d1120 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd223c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c0d1530 con 0x7fd21c154950 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd223c00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c162460 con 0x7fd21c161b10 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd221200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd21c0d1120 0x7fd21c070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56482/0 (socket says 172.21.15.123:56482) 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd221200640 1 -- 172.21.15.123:0/1303095295 learned_addr learned my addr 172.21.15.123:0/1303095295 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.880+0000 7fd221c00640 1 --2- 172.21.15.123:0/1303095295 >> v2:172.21.15.123:3300/0 conn(0x7fd21c154950 0x7fd21c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd21be00640 1 --2- 172.21.15.123:0/1303095295 >> v2:172.21.15.146:3300/0 conn(0x7fd21c161b10 0x7fd21c161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd21b400640 1 -- 172.21.15.123:0/1303095295 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 339349697 0 0) 0x7fd21c071600 con 0x7fd21c0d1120 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd21b400640 1 -- 172.21.15.123:0/1303095295 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd200003520 con 0x7fd21c0d1120 2024-04-25T19:08:42.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd221c00640 1 -- 172.21.15.123:0/1303095295 >> v2:172.21.15.146:3300/0 conn(0x7fd21c161b10 msgr2=0x7fd21c161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:42.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd221c00640 1 --2- 172.21.15.123:0/1303095295 >> v2:172.21.15.146:3300/0 conn(0x7fd21c161b10 0x7fd21c161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd221c00640 1 -- 172.21.15.123:0/1303095295 >> v1:172.21.15.123:6789/0 conn(0x7fd21c0d1120 legacy=0x7fd21c070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.881+0000 7fd221c00640 1 -- 172.21.15.123:0/1303095295 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0aec30 con 0x7fd21c154950 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.882+0000 7fd221c00640 1 --2- 172.21.15.123:0/1303095295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c154950 0x7fd21c154d60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0b1260 tx=0x7fd21402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ef3683740bb8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.882+0000 7fd21b400640 1 -- 172.21.15.123:0/1303095295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214037070 con 0x7fd21c154950 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.882+0000 7fd21b400640 1 -- 172.21.15.123:0/1303095295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21402add0 con 0x7fd21c154950 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.882+0000 7fd21b400640 1 -- 172.21.15.123:0/1303095295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214032660 con 0x7fd21c154950 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 -- 172.21.15.123:0/1303095295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c154950 msgr2=0x7fd21c154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 --2- 172.21.15.123:0/1303095295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c154950 0x7fd21c154d60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0b1260 tx=0x7fd21402a080 comp rx=0 tx=0).stop 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 -- 172.21.15.123:0/1303095295 shutdown_connections 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 --2- 172.21.15.123:0/1303095295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c154950 0x7fd21c154d60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 --2- 172.21.15.123:0/1303095295 >> v2:172.21.15.146:3300/0 conn(0x7fd21c161b10 0x7fd21c161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 -- 172.21.15.123:0/1303095295 >> 172.21.15.123:0/1303095295 conn(0x7fd21c0d1ae0 msgr2=0x7fd21c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 -- 172.21.15.123:0/1303095295 shutdown_connections 2024-04-25T19:08:42.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.883+0000 7fd223c00640 1 -- 172.21.15.123:0/1303095295 wait complete. 2024-04-25T19:08:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.885+0000 7fd223c00640 1 Processor -- start 2024-04-25T19:08:42.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.886+0000 7fd223c00640 1 -- start start 2024-04-25T19:08:42.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd21c0d1120 0x7fd21c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd21c154950 0x7fd21c1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0af4d0 con 0x7fd21c161b10 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0af1f0 con 0x7fd21c0d1120 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd223c00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c1552a0 con 0x7fd21c154950 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd221c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd221c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48274/0 (socket says 172.21.15.123:48274) 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.887+0000 7fd221c00640 1 -- 172.21.15.123:0/4214417128 learned_addr learned my addr 172.21.15.123:0/4214417128 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221200640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd21c0d1120 0x7fd21c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd21be00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd21c154950 0x7fd21c1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 -- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd21c154950 msgr2=0x7fd21c1f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd21c154950 0x7fd21c1f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 -- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd21c0d1120 msgr2=0x7fd21c1541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd21c0d1120 0x7fd21c1541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 -- 172.21.15.123:0/4214417128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c1f8920 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.888+0000 7fd221c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd214009890 tx=0x7fd2140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.889+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214037040 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.889+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214042070 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.889+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21403d680 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.889+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c1f4830 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.889+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1f4df0 con 0x7fd21c161b10 2024-04-25T19:08:42.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.890+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c1f8690 con 0x7fd21c161b10 2024-04-25T19:08:42.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.892+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd21403d820 con 0x7fd21c161b10 2024-04-25T19:08:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.892+0000 7fd218c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 0x7fd1fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.893+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fd2140b6180 con 0x7fd21c161b10 2024-04-25T19:08:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.893+0000 7fd221200640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 0x7fd1fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.894+0000 7fd221200640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 0x7fd1fc0769f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005c00 tx=0x7fd20c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:42.897+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd214080b60 con 0x7fd21c161b10 2024-04-25T19:08:43.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.103+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fd21c076a40 con 0x7fd1fc0744e0 2024-04-25T19:08:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.104+0000 7fd218c00640 1 -- 172.21.15.123:0/4214417128 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd21c076a40 con 0x7fd1fc0744e0 2024-04-25T19:08:43.106 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T19:08:43.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.106+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 msgr2=0x7fd1fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.106+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 0x7fd1fc0769f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005c00 tx=0x7fd20c009040 comp rx=0 tx=0).stop 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.106+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 msgr2=0x7fd21c06c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.106+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd214009890 tx=0x7fd2140046c0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 shutdown_connections 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fd1fc0744e0 0x7fd1fc0769f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c161b10 0x7fd21c06c600 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd21c154950 0x7fd21c1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 --2- 172.21.15.123:0/4214417128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd21c0d1120 0x7fd21c1541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 >> 172.21.15.123:0/4214417128 conn(0x7fd21c0d1ae0 msgr2=0x7fd21c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 shutdown_connections 2024-04-25T19:08:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.107+0000 7fd223c00640 1 -- 172.21.15.123:0/4214417128 wait complete. 2024-04-25T19:08:43.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:08:43.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.405+0000 7fba6b200640 1 Processor -- start 2024-04-25T19:08:43.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.406+0000 7fba6b200640 1 -- start start 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6b200640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fba6c06fe20 0x7fba6c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6b200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba6c06ded0 con 0x7fba6c15e2f0 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6b200640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fba6c15ee10 con 0x7fba6c06fe20 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6b200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fba6c15f070 con 0x7fba6c06d580 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6a800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fba6c15e2f0 0x7fba6c15e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56492/0 (socket says 172.21.15.123:56492) 2024-04-25T19:08:43.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.408+0000 7fba6a800640 1 -- 172.21.15.123:0/1543850278 learned_addr learned my addr 172.21.15.123:0/1543850278 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-25T19:08:43.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69e00640 1 --2- 172.21.15.123:0/1543850278 >> v2:172.21.15.146:3300/0 conn(0x7fba6c06fe20 0x7fba6c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba61400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960957397 0 0) 0x7fba64001490 con 0x7fba6c15e2f0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba50003520 con 0x7fba6c15e2f0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2157282464 0 0) 0x7fba50003520 con 0x7fba6c15e2f0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 msgr2=0x7fba6c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba6c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 >> v2:172.21.15.146:3300/0 conn(0x7fba6c06fe20 msgr2=0x7fba6c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 --2- 172.21.15.123:0/1543850278 >> v2:172.21.15.146:3300/0 conn(0x7fba6c06fe20 0x7fba6c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.409+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c06f340 con 0x7fba6c15e2f0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 479039550 0 0) 0x7fba64001bc0 con 0x7fba6c15e2f0 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 >> v1:172.21.15.123:6789/0 conn(0x7fba6c15e2f0 legacy=0x7fba6c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba50003f30 0x7fba50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba50005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c15ee10 con 0x7fba50004d40 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c15f070 con 0x7fba50003f30 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fba5000bb50 con 0x7fba6c06d580 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba6a800640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba50005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.410+0000 7fba69e00640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba6a800640 1 -- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 msgr2=0x7fba500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba61400640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba50003f30 0x7fba50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba6a800640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c06d580 0x7fba500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba6a800640 1 -- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba50003f30 msgr2=0x7fba50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba6a800640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba50003f30 0x7fba50004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.411+0000 7fba6a800640 1 -- 172.21.15.123:0/1543850278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba50003520 con 0x7fba50004d40 2024-04-25T19:08:43.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.412+0000 7fba6a800640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba50005150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fba6400aa50 tx=0x7fba50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189d630ba72e205f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.412+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6400a670 con 0x7fba50004d40 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.412+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6402f070 con 0x7fba50004d40 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.412+0000 7fba69400640 1 -- 172.21.15.123:0/1543850278 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64034470 con 0x7fba50004d40 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 -- 172.21.15.123:0/1543850278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 msgr2=0x7fba50005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 --2- 172.21.15.123:0/1543850278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba50005150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fba6400aa50 tx=0x7fba50005830 comp rx=0 tx=0).stop 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6a800640 1 -- 172.21.15.123:0/1543850278 reap_dead start 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 -- 172.21.15.123:0/1543850278 shutdown_connections 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 -- 172.21.15.123:0/1543850278 >> 172.21.15.123:0/1543850278 conn(0x7fba6c073200 msgr2=0x7fba6c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 -- 172.21.15.123:0/1543850278 shutdown_connections 2024-04-25T19:08:43.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.413+0000 7fba6b200640 1 -- 172.21.15.123:0/1543850278 wait complete. 2024-04-25T19:08:43.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.415+0000 7fba6b200640 1 Processor -- start 2024-04-25T19:08:43.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.415+0000 7fba6b200640 1 -- start start 2024-04-25T19:08:43.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c1f6490 0x7fba6c1f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba6c1f6de0 0x7fba6c0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c15bbf0 con 0x7fba50004d40 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c06f900 con 0x7fba6c1f6de0 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba6b200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fba6c06d990 con 0x7fba6c1f6490 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.417+0000 7fba69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48296/0 (socket says 172.21.15.123:48296) 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 -- 172.21.15.123:0/2567208664 learned_addr learned my addr 172.21.15.123:0/2567208664 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 -- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c1f6490 msgr2=0x7fba6c1f68a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c1f6490 0x7fba6c1f68a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 -- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba6c1f6de0 msgr2=0x7fba6c0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba6c1f6de0 0x7fba6c0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 -- 172.21.15.123:0/2567208664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c0cd830 con 0x7fba50004d40 2024-04-25T19:08:43.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba69e00640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fba5c011600 tx=0x7fba5c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c019070 con 0x7fba50004d40 2024-04-25T19:08:43.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c004440 con 0x7fba50004d40 2024-04-25T19:08:43.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.418+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6c1e7c40 con 0x7fba50004d40 2024-04-25T19:08:43.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.419+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c003850 con 0x7fba50004d40 2024-04-25T19:08:43.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.419+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6c1e8430 con 0x7fba50004d40 2024-04-25T19:08:43.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.420+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6c0699b0 con 0x7fba50004d40 2024-04-25T19:08:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.421+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fba5c01c070 con 0x7fba50004d40 2024-04-25T19:08:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.423+0000 7fba62a00640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.423+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fba5c0951a0 con 0x7fba50004d40 2024-04-25T19:08:43.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.423+0000 7fba61400640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.424+0000 7fba61400640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 0x7fba480769f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba6c15ae90 tx=0x7fba54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.428+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba6c0699b0 con 0x7fba50004d40 2024-04-25T19:08:43.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.627+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 --> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba6c070a00 con 0x7fba480744e0 2024-04-25T19:08:43.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.629+0000 7fba62a00640 1 -- 172.21.15.123:0/2567208664 <== mgr.4104 v2:172.21.15.123:6832/1011550908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13381 (secure 0 0 0) 0x7fba6c070a00 con 0x7fba480744e0 2024-04-25T19:08:43.631 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:43.631 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-25T19:08:43.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 msgr2=0x7fba480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 0x7fba480769f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba6c15ae90 tx=0x7fba54008040 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 msgr2=0x7fba6c1e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fba5c011600 tx=0x7fba5c011ac0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 shutdown_connections 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba6c1f6de0 0x7fba6c0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.632+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fba6c1f6490 0x7fba6c1f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.633+0000 7fba6b200640 1 --2- 172.21.15.123:0/2567208664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba50004d40 0x7fba6c1e7700 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.633+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 >> 172.21.15.123:0/2567208664 conn(0x7fba6c073200 msgr2=0x7fba6c161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.633+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 shutdown_connections 2024-04-25T19:08:43.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.633+0000 7fba6b200640 1 -- 172.21.15.123:0/2567208664 wait complete. 2024-04-25T19:08:43.659 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-04-25T19:08:43.089912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225592,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749128,"statfs":{"total":773094113280,"available":772863107072,"internally_reserved":0,"allocated":5382144,"data_stored":2993480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003357"},"pg_stats":[{"pgid":"1.0","version":"45'74","reported_seq":166,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T19:08:38.731709+0000","last_change":"2024-04-25T19:08:38.731853+0000","last_active":"2024-04-25T19:08:38.731709+0000","last_peered":"2024-04-25T19:08:38.731709+0000","last_clean":"2024-04-25T19:08:38.731709+0000","last_became_active":"2024-04-25T19:02:29.656265+0000","last_became_peered":"2024-04-25T19:02:29.656265+0000","last_unstale":"2024-04-25T19:08:38.731709+0000","last_undegraded":"2024-04-25T19:08:38.731709+0000","last_fullsized":"2024-04-25T19:08:38.731709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'74","last_scrub_stamp":"2024-04-25T19:08:38.731450+0000","last_deep_scrub":"45'74","last_deep_scrub_stamp":"2024-04-25T19:08:38.731450+0000","last_clean_scrub_stamp":"2024-04-25T19:08:38.731450+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:54.591104+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1118208,"data_stored":816961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":226593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:08:43.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T19:08:43.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.911+0000 7fade7e00640 1 Processor -- start 2024-04-25T19:08:43.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.911+0000 7fade7e00640 1 -- start start 2024-04-25T19:08:43.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade815df10 0x7fade80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fade80b0700 0x7fade8162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fade806d580 0x7fade806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fade8070f40 con 0x7fade806d580 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fade8161e90 con 0x7fade80b0700 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.912+0000 7fade7e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fade80d1430 con 0x7fade815df10 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fade806d580 0x7fade806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fade806d580 0x7fade806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48310/0 (socket says 172.21.15.123:48310) 2024-04-25T19:08:43.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 -- 172.21.15.123:0/1064554056 learned_addr learned my addr 172.21.15.123:0/1064554056 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 -- 172.21.15.123:0/1064554056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade815df10 msgr2=0x7fade80702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fadde000640 1 --2- 172.21.15.123:0/1064554056 >> v2:172.21.15.146:3300/0 conn(0x7fade80b0700 0x7fade8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade6a00640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade815df10 0x7fade80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade815df10 0x7fade80702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 -- 172.21.15.123:0/1064554056 >> v2:172.21.15.146:3300/0 conn(0x7fade80b0700 msgr2=0x7fade8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 --2- 172.21.15.123:0/1064554056 >> v2:172.21.15.146:3300/0 conn(0x7fade80b0700 0x7fade8162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.913+0000 7fade7400640 1 -- 172.21.15.123:0/1064554056 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade806ded0 con 0x7fade806d580 2024-04-25T19:08:43.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.914+0000 7fade7400640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade806d580 0x7fade806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fade80b69c0 tx=0x7fade002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b2104c6bd33afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.915+0000 7fade6000640 1 -- 172.21.15.123:0/1064554056 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0036070 con 0x7fade806d580 2024-04-25T19:08:43.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.915+0000 7fade6000640 1 -- 172.21.15.123:0/1064554056 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade0031600 con 0x7fade806d580 2024-04-25T19:08:43.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.915+0000 7fade6000640 1 -- 172.21.15.123:0/1064554056 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00319d0 con 0x7fade806d580 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.915+0000 7fade7e00640 1 -- 172.21.15.123:0/1064554056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade806d580 msgr2=0x7fade806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.915+0000 7fade7e00640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade806d580 0x7fade806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fade80b69c0 tx=0x7fade002a6a0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 -- 172.21.15.123:0/1064554056 shutdown_connections 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade806d580 0x7fade806d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 --2- 172.21.15.123:0/1064554056 >> v2:172.21.15.146:3300/0 conn(0x7fade80b0700 0x7fade8162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 --2- 172.21.15.123:0/1064554056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade815df10 0x7fade80702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 -- 172.21.15.123:0/1064554056 >> 172.21.15.123:0/1064554056 conn(0x7fade8073090 msgr2=0x7fade80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 -- 172.21.15.123:0/1064554056 shutdown_connections 2024-04-25T19:08:43.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.916+0000 7fade7e00640 1 -- 172.21.15.123:0/1064554056 wait complete. 2024-04-25T19:08:43.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.918+0000 7fade7e00640 1 Processor -- start 2024-04-25T19:08:43.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.918+0000 7fade7e00640 1 -- start start 2024-04-25T19:08:43.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fade806d580 0x7fade814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade80b0700 0x7fade814a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fade8161b20 con 0x7fade815df10 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fade8070c60 con 0x7fade806d580 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7e00640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fade8162860 con 0x7fade80b0700 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48318/0 (socket says 172.21.15.123:48318) 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.920+0000 7fade7400640 1 -- 172.21.15.123:0/2889374710 learned_addr learned my addr 172.21.15.123:0/2889374710 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade6a00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fade806d580 0x7fade814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 -- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade80b0700 msgr2=0x7fade814a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade80b0700 0x7fade814a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 -- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fade806d580 msgr2=0x7fade814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fade806d580 0x7fade814cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 -- 172.21.15.123:0/2889374710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade815bea0 con 0x7fade815df10 2024-04-25T19:08:43.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7400640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fade0009f40 tx=0x7fade0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0036050 con 0x7fade815df10 2024-04-25T19:08:43.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade0042070 con 0x7fade815df10 2024-04-25T19:08:43.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0009510 con 0x7fade815df10 2024-04-25T19:08:43.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade815c180 con 0x7fade815df10 2024-04-25T19:08:43.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.921+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade815c740 con 0x7fade815df10 2024-04-25T19:08:43.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.922+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade8050740 con 0x7fade815df10 2024-04-25T19:08:43.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.924+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fade0007bd0 con 0x7fade815df10 2024-04-25T19:08:43.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.925+0000 7faddf400640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 0x7fadc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.925+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fade00b5ff0 con 0x7fade815df10 2024-04-25T19:08:43.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.925+0000 7fade6a00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 0x7fadc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.926+0000 7fade6a00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 0x7fadc40769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fade81618e0 tx=0x7fadd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:43.930+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fade806ded0 con 0x7fade815df10 2024-04-25T19:08:44.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.129+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fade8074190 con 0x7fade815df10 2024-04-25T19:08:44.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.138+0000 7faddf400640 1 -- 172.21.15.123:0/2889374710 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fade0080940 con 0x7fade815df10 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.141+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 msgr2=0x7fadc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.141+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 0x7fadc40769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fade81618e0 tx=0x7fadd8008040 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.141+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 msgr2=0x7fade815cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.141+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fade0009f40 tx=0x7fade0004a40 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 shutdown_connections 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7fadc40744b0 0x7fadc40769c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade815df10 0x7fade815cf40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fade80b0700 0x7fade814a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 --2- 172.21.15.123:0/2889374710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fade806d580 0x7fade814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.142+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 >> 172.21.15.123:0/2889374710 conn(0x7fade8073090 msgr2=0x7fade80b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:44.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.143+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 shutdown_connections 2024-04-25T19:08:44.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.144+0000 7fade7e00640 1 -- 172.21.15.123:0/2889374710 wait complete. 2024-04-25T19:08:44.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:08:44.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:08:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.441+0000 7f38bf200640 1 Processor -- start 2024-04-25T19:08:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.441+0000 7f38bf200640 1 -- start start 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.443+0000 7f38bf200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006fe20 0x7f38c015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.443+0000 7f38bf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38c006d580 0x7f38c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.443+0000 7f38bf200640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c006ded0 con 0x7f38c015e2f0 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.443+0000 7f38bf200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f38c015ee10 con 0x7f38c006d580 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.443+0000 7f38bf200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f38c015f070 con 0x7f38c006fe20 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38c006d580 0x7f38c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38c006d580 0x7f38c006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48332/0 (socket says 172.21.15.123:48332) 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 -- 172.21.15.123:0/2213829615 learned_addr learned my addr 172.21.15.123:0/2213829615 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 -- 172.21.15.123:0/2213829615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006fe20 msgr2=0x7f38c015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 --2- 172.21.15.123:0/2213829615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006fe20 0x7f38c015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 -- 172.21.15.123:0/2213829615 >> v1:172.21.15.146:6789/0 conn(0x7f38c015e2f0 legacy=0x7f38c015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.444+0000 7f38bd400640 1 -- 172.21.15.123:0/2213829615 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c006f340 con 0x7f38c006d580 2024-04-25T19:08:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.445+0000 7f38bd400640 1 --2- 172.21.15.123:0/2213829615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c006d580 0x7f38c006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009870 tx=0x7f38ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944e5dc1f8a3c3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bca00640 1 -- 172.21.15.123:0/2213829615 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037070 con 0x7f38c006d580 2024-04-25T19:08:44.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bca00640 1 -- 172.21.15.123:0/2213829615 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f38ac032440 con 0x7f38c006d580 2024-04-25T19:08:44.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bca00640 1 -- 172.21.15.123:0/2213829615 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac032830 con 0x7f38c006d580 2024-04-25T19:08:44.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bf200640 1 -- 172.21.15.123:0/2213829615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c006d580 msgr2=0x7f38c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bf200640 1 --2- 172.21.15.123:0/2213829615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c006d580 0x7f38c006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009870 tx=0x7f38ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.446+0000 7f38bf200640 1 -- 172.21.15.123:0/2213829615 shutdown_connections 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.447+0000 7f38bf200640 1 --2- 172.21.15.123:0/2213829615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c006d580 0x7f38c006d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.447+0000 7f38bf200640 1 --2- 172.21.15.123:0/2213829615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006fe20 0x7f38c015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.447+0000 7f38bf200640 1 -- 172.21.15.123:0/2213829615 >> 172.21.15.123:0/2213829615 conn(0x7f38c0073200 msgr2=0x7f38c0073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.447+0000 7f38bf200640 1 -- 172.21.15.123:0/2213829615 shutdown_connections 2024-04-25T19:08:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.447+0000 7f38bf200640 1 -- 172.21.15.123:0/2213829615 wait complete. 2024-04-25T19:08:44.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.449+0000 7f38bf200640 1 Processor -- start 2024-04-25T19:08:44.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.449+0000 7f38bf200640 1 -- start start 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.450+0000 7f38bf200640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006d580 0x7f38c0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bf200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38c006fe20 0x7f38c00cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38c015bbf0 con 0x7f38c015e2f0 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bf200640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f38c006f900 con 0x7f38c006fe20 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bf200640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f38c00ce500 con 0x7f38c006d580 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:44.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48340/0 (socket says 172.21.15.123:48340) 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 -- 172.21.15.123:0/2188847863 learned_addr learned my addr 172.21.15.123:0/2188847863 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 -- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006d580 msgr2=0x7f38c0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38bde00640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006d580 0x7f38c0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006d580 0x7f38c0159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 -- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38c006fe20 msgr2=0x7f38c00cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38c006fe20 0x7f38c00cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.451+0000 7f38be800640 1 -- 172.21.15.123:0/2188847863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c015a280 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38be800640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b9e0 tx=0x7f38b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8019070 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f38b800cb20 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c00c9530 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8014440 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.452+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c00cd180 con 0x7f38c015e2f0 2024-04-25T19:08:44.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.453+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c00ccbf0 con 0x7f38c015e2f0 2024-04-25T19:08:44.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.455+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f38b801c070 con 0x7f38c015e2f0 2024-04-25T19:08:44.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.456+0000 7f38b6000640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 0x7f38a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:44.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.456+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f38b8094e10 con 0x7f38c015e2f0 2024-04-25T19:08:44.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.457+0000 7f38bde00640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 0x7f38a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:44.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.457+0000 7f38bde00640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 0x7f38a00769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38c00742c0 tx=0x7f38b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:44.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.462+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38b8098030 con 0x7f38c015e2f0 2024-04-25T19:08:44.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.696+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38c0050740 con 0x7f38c015e2f0 2024-04-25T19:08:44.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.697+0000 7f38b6000640 1 -- 172.21.15.123:0/2188847863 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3878 (secure 0 0 0) 0x7f38c015a280 con 0x7f38c015e2f0 2024-04-25T19:08:44.699 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-25T19:08:44.699 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3314871011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3314871011},{"type":"v1","addr":"172.21.15.146:6835","nonce":3314871011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3194892084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3194892084},{"type":"v1","addr":"172.21.15.123:6837","nonce":3194892084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/631553705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":631553705},{"type":"v1","addr":"172.21.15.146:6834","nonce":631553705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1551063055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1551063055},{"type":"v1","addr":"172.21.15.123:6835","nonce":1551063055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-25T19:08:44.699 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 20 2024-04-25T19:08:44.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.699+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 msgr2=0x7f38a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.699+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 0x7f38a00769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38c00742c0 tx=0x7f38b0008040 comp rx=0 tx=0).stop 2024-04-25T19:08:44.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.699+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 msgr2=0x7f38c015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:44.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.699+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b9e0 tx=0x7f38b800bea0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 shutdown_connections 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:6832/1011550908,v1:172.21.15.123:6833/1011550908] conn(0x7f38a00744b0 0x7f38a00769c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38c015e2f0 0x7f38c015cef0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38c006fe20 0x7f38c00cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 --2- 172.21.15.123:0/2188847863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38c006d580 0x7f38c0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:44.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 >> 172.21.15.123:0/2188847863 conn(0x7f38c0073200 msgr2=0x7f38c013fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:44.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 shutdown_connections 2024-04-25T19:08:44.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-25T19:08:44.700+0000 7f38bf200640 1 -- 172.21.15.123:0/2188847863 wait complete. 2024-04-25T19:08:44.726 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T19:08:44.726 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T19:08:44.727 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T19:08:44.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:44.775 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T19:08:44.775 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T19:08:44.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:44.813 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T19:08:44.813 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T19:08:44.813 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:44.854 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T19:08:44.854 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T19:08:44.854 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:44.939 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T19:08:44.939 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T19:08:44.940 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T19:08:44.940 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.002 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T19:08:45.002 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T19:08:45.002 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.084 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T19:08:45.084 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T19:08:45.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.136 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T19:08:45.136 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T19:08:45.136 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.211 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T19:08:45.211 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T19:08:45.211 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.306 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T19:08:45.306 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T19:08:45.306 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.381 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T19:08:45.381 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T19:08:45.381 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.450 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T19:08:45.451 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T19:08:45.451 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.511 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T19:08:45.511 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T19:08:45.511 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T19:08:45.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.603 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T19:08:45.604 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T19:08:45.604 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.687 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T19:08:45.687 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T19:08:45.688 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T19:08:45.688 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.760 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T19:08:45.761 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T19:08:45.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:45.792 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T19:08:45.792 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T19:08:45.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:08:48.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T19:08:51.895 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T19:08:51.895 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T19:08:51.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-04-25T19:08:52.005 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:52.006 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T19:08:52.387 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:52.387 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T19:08:52.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:52.529 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T19:08:52.674 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:52.674 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T19:08:52.846 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:08:52.847 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T19:08:53.129 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:08:53.129 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T19:08:53.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T19:08:53.264 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:08:53.264 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T19:08:53.386 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:08:53.386 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T19:08:53.588 INFO:tasks.ceph:Archiving mon data... 2024-04-25T19:08:53.589 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/data/mon.a.tgz 2024-04-25T19:08:53.589 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-25T19:08:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.sEnvaAHbBR 2024-04-25T19:08:53.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sEnvaAHbBR 2024-04-25T19:08:53.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.sEnvaAHbBR 2024-04-25T19:08:53.905 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.sEnvaAHbBR is 349KB 2024-04-25T19:08:54.001 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.sEnvaAHbBR 2024-04-25T19:08:54.051 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/data/mon.b.tgz 2024-04-25T19:08:54.052 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-04-25T19:08:54.109 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.eIstnBYZ6S 2024-04-25T19:08:54.110 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eIstnBYZ6S 2024-04-25T19:08:54.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.eIstnBYZ6S 2024-04-25T19:08:54.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:54.393 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:54.484 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.eIstnBYZ6S is 461KB 2024-04-25T19:08:54.586 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.eIstnBYZ6S 2024-04-25T19:08:54.644 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/data/mon.c.tgz 2024-04-25T19:08:54.644 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-04-25T19:08:54.696 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.X0K5erX6iN 2024-04-25T19:08:54.697 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.X0K5erX6iN 2024-04-25T19:08:54.883 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.X0K5erX6iN 2024-04-25T19:08:55.009 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.X0K5erX6iN is 461KB 2024-04-25T19:08:55.119 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.X0K5erX6iN 2024-04-25T19:08:55.174 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T19:08:55.174 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T19:08:55.182 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T19:08:55.321 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T19:08:55.379 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T19:08:55.379 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T19:08:55.380 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi123/crash 2024-04-25T19:08:55.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T19:08:55.470 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi146/crash 2024-04-25T19:08:55.471 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T19:08:55.554 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T19:08:55.555 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:55.562 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:55.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:55.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T19:08:55.844 INFO:tasks.ceph:Compressing logs... 2024-04-25T19:08:55.844 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:08:55.851 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:08:55.926 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48586.log 2024-04-25T19:08:55.926 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45507.log 2024-04-25T19:08:55.927 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.0.48586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48586.log.gz 2024-04-25T19:08:55.927 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-04-25T19:08:55.929 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40544.log 2024-04-25T19:08:55.929 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45507.log.gz 2024-04-25T19:08:55.930 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106326.log 2024-04-25T19:08:55.930 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-04-25T19:08:55.931 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-04-25T19:08:55.932 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40544.log.gz 2024-04-25T19:08:55.932 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33341.log 2024-04-25T19:08:55.933 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105863.log 2024-04-25T19:08:55.933 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106326.log.gz 2024-04-25T19:08:55.933 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105527.log 2024-04-25T19:08:55.933 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-04-25T19:08:55.937 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T19:08:55.937 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33341.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33341.log.gz 2024-04-25T19:08:55.937 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T19:08:55.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T19:08:55.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T19:08:55.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T19:08:55.938 INFO:teuthology.orchestra.run.smithi146.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104666.log 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105863.log.gz 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106368.log 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105527.log.gz 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104666.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104666.log.gz 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45102.log 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45692.log 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106368.log.gz 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45144.log 2024-04-25T19:08:55.939 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45102.log.gz 2024-04-25T19:08:55.941 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45692.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45692.log.gz 2024-04-25T19:08:55.941 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39292.log 2024-04-25T19:08:55.941 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-04-25T19:08:55.941 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45144.log.gz 2024-04-25T19:08:55.943 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33421.log 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33216.log 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33421.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33421.log.gz 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33381.log 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33216.log.gz 2024-04-25T19:08:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.38951.log 2024-04-25T19:08:55.946 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T19:08:55.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33381.log: /var/log/ceph/ceph-client.3.38951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.38951.log.gz 2024-04-25T19:08:55.947 INFO:teuthology.orchestra.run.smithi146.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33381.log.gz 2024-04-25T19:08:55.947 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39292.log: gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39292.log.gz /var/log/ceph/ceph-client.admin.45228.log 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42402.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46917.log 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45228.log.gz 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104876.log 2024-04-25T19:08:55.948 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-04-25T19:08:55.949 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46917.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46917.log.gz 2024-04-25T19:08:55.949 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45465.log 2024-04-25T19:08:55.949 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106241.log 2024-04-25T19:08:55.949 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104876.log.gz 2024-04-25T19:08:55.949 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105148.log 2024-04-25T19:08:55.950 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T19:08:55.950 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33261.log 2024-04-25T19:08:55.951 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45465.log.gz 2024-04-25T19:08:55.951 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47043.log 2024-04-25T19:08:55.951 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106241.log.gz 2024-04-25T19:08:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2024-04-25T19:08:55.952 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105148.log.gz 2024-04-25T19:08:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-04-25T19:08:55.952 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47043.log.gz 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33261.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33261.log.gz 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33461.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38536.log 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39420.log 2024-04-25T19:08:55.954 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-04-25T19:08:55.956 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33170.log 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47371.log 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39420.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39420.log.gz 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46833.log 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-04-25T19:08:55.957 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47371.log.gz 2024-04-25T19:08:55.958 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46833.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46833.log.gz 2024-04-25T19:08:55.958 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46623.log 2024-04-25T19:08:55.959 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.38536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.38536.log.gz 2024-04-25T19:08:55.959 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T19:08:55.959 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33301.log 2024-04-25T19:08:55.959 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33170.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33170.log.gz 2024-04-25T19:08:55.959 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T19:08:55.960 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33301.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T19:08:55.960 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33301.log.gz 2024-04-25T19:08:55.961 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39078.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T19:08:55.961 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-04-25T19:08:55.961 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-04-25T19:08:55.961 INFO:teuthology.orchestra.run.smithi123.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46623.log.gz 2024-04-25T19:08:55.961 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104750.log 2024-04-25T19:08:55.963 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39035.log 2024-04-25T19:08:55.963 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-04-25T19:08:55.963 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105020.log 2024-04-25T19:08:55.963 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104750.log.gz 2024-04-25T19:08:55.964 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39035.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39035.log.gz 2024-04-25T19:08:55.964 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-04-25T19:08:55.965 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104488.log 2024-04-25T19:08:55.965 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105020.log.gz 2024-04-25T19:08:55.966 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48398.log 2024-04-25T19:08:55.966 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-04-25T19:08:55.967 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46875.log 2024-04-25T19:08:55.967 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104488.log.gz 2024-04-25T19:08:55.968 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-04-25T19:08:55.968 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48398.log.gz 2024-04-25T19:08:55.969 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104978.log 2024-04-25T19:08:55.970 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46875.log.gz 2024-04-25T19:08:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105779.log 2024-04-25T19:08:55.971 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-04-25T19:08:55.972 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104792.log 2024-04-25T19:08:55.972 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104978.log.gz 2024-04-25T19:08:55.973 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40628.log 2024-04-25T19:08:55.973 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105779.log.gz 2024-04-25T19:08:55.974 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47127.log 2024-04-25T19:08:55.974 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104792.log.gz 2024-04-25T19:08:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105653.log 2024-04-25T19:08:55.975 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40628.log.gz 2024-04-25T19:08:55.976 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41614.log 2024-04-25T19:08:55.976 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47127.log.gz 2024-04-25T19:08:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105653.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T19:08:55.977 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105653.log.gz 2024-04-25T19:08:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-04-25T19:08:55.978 INFO:teuthology.orchestra.run.smithi123.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41614.log.gz 2024-04-25T19:08:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33891.log 2024-04-25T19:08:55.981 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2024-04-25T19:08:55.981 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-04-25T19:08:55.981 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T19:08:55.982 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.33891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105905.log 2024-04-25T19:08:55.982 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33891.log.gz 2024-04-25T19:08:55.983 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-04-25T19:08:55.983 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2024-04-25T19:08:55.984 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-04-25T19:08:55.984 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105905.log.gz 2024-04-25T19:08:55.985 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38993.log 2024-04-25T19:08:55.985 INFO:teuthology.orchestra.run.smithi123.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-04-25T19:08:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41142.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-04-25T19:08:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38816.log 2024-04-25T19:08:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-04-25T19:08:55.988 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38993.log.gz 2024-04-25T19:08:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46167.log 2024-04-25T19:08:55.989 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2024-04-25T19:08:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-04-25T19:08:55.990 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-04-25T19:08:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105695.log 2024-04-25T19:08:55.991 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46167.log.gz 2024-04-25T19:08:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-04-25T19:08:55.992 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-04-25T19:08:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-25T19:08:55.993 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105695.log.gz 2024-04-25T19:08:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103737.log 2024-04-25T19:08:55.994 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-04-25T19:08:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-04-25T19:08:55.996 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-25T19:08:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103737.log.gz 2024-04-25T19:08:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T19:08:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41606.log 2024-04-25T19:08:55.998 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-04-25T19:08:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 2024-04-25T19:08:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2024-04-25T19:08:56.000 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41606.log.gz 2024-04-25T19:08:56.001 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-04-25T19:08:56.001 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-04-25T19:08:56.002 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-04-25T19:08:56.002 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2024-04-25T19:08:56.003 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44976.log 2024-04-25T19:08:56.003 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-04-25T19:08:56.005 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43656.log 2024-04-25T19:08:56.005 INFO:teuthology.orchestra.run.smithi123.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-04-25T19:08:56.006 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45060.log 2024-04-25T19:08:56.006 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44976.log.gz 2024-04-25T19:08:56.007 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-25T19:08:56.007 INFO:teuthology.orchestra.run.smithi123.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43656.log.gz 2024-04-25T19:08:56.008 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106157.log 2024-04-25T19:08:56.008 INFO:teuthology.orchestra.run.smithi123.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45060.log.gz 2024-04-25T19:08:56.009 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T19:08:56.010 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-04-25T19:08:56.010 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-25T19:08:56.011 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106157.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T19:08:56.011 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106157.log.gz 2024-04-25T19:08:56.012 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-04-25T19:08:56.012 INFO:teuthology.orchestra.run.smithi123.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-04-25T19:08:56.013 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/xfstests-dev-results/results/check.log 2024-04-25T19:08:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41268.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-04-25T19:08:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-04-25T19:08:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-04-25T19:08:56.016 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/xfstests-dev-results/results/check.log: /var/log/ceph/ceph-client.admin.41016.log: -3.0% -- replaced with /var/log/ceph/xfstests-dev-results/results/check.log.gz 2024-04-25T19:08:56.017 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-04-25T19:08:56.017 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-04-25T19:08:56.018 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-04-25T19:08:56.019 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39204.log: /var/log/ceph/ceph-client.admin.47709.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gz 2024-04-25T19:08:56.019 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-04-25T19:08:56.019 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-04-25T19:08:56.020 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-04-25T19:08:56.021 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.43154.log: /var/log/ceph/ceph-client.admin.46329.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-04-25T19:08:56.022 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-04-25T19:08:56.022 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-04-25T19:08:56.022 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-04-25T19:08:56.023 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-04-25T19:08:56.024 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-04-25T19:08:56.024 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46959.log 2024-04-25T19:08:56.025 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-04-25T19:08:56.026 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39504.log: /var/log/ceph/ceph-client.admin.46959.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-04-25T19:08:56.026 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46455.log 2024-04-25T19:08:56.026 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46959.log.gz 2024-04-25T19:08:56.027 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-04-25T19:08:56.028 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-04-25T19:08:56.029 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46455.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46455.log.gz 2024-04-25T19:08:56.029 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105063.log 2024-04-25T19:08:56.030 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T19:08:56.031 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44304.log: /var/log/ceph/ceph-client.admin.105063.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2024-04-25T19:08:56.031 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39248.log 89.4% 2024-04-25T19:08:56.031 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105063.log.gz 2024-04-25T19:08:56.032 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T19:08:56.033 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39248.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2024-04-25T19:08:56.033 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48038.log 2024-04-25T19:08:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-04-25T19:08:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48038.log: gzip -5 88.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.48038.log.gz -- 2024-04-25T19:08:56.036 INFO:teuthology.orchestra.run.smithi123.stderr: /var/log/ceph/ceph-client.admin.104918.log 2024-04-25T19:08:56.037 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45270.log 2024-04-25T19:08:56.038 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-04-25T19:08:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104918.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104918.log.gz 2024-04-25T19:08:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-04-25T19:08:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45270.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45270.log.gz 2024-04-25T19:08:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-04-25T19:08:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46209.log 2024-04-25T19:08:56.042 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-04-25T19:08:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47583.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T19:08:56.043 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-04-25T19:08:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46209.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46209.log.gz 2024-04-25T19:08:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26832.log 2024-04-25T19:08:56.045 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44514.log 2024-04-25T19:08:56.047 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.tmp-client.admin.26832.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26832.log.gz 2024-04-25T19:08:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39630.log 2024-04-25T19:08:56.049 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz 2024-04-25T19:08:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39630.log.gz 2024-04-25T19:08:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-04-25T19:08:56.051 INFO:teuthology.orchestra.run.smithi146.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T19:08:56.052 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104708.log 2024-04-25T19:08:56.053 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-04-25T19:08:56.054 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104708.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104708.log.gz 2024-04-25T19:08:56.054 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47908.log 2024-04-25T19:08:56.056 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105251.log 2024-04-25T19:08:56.057 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47908.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47908.log.gz 2024-04-25T19:08:56.057 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-04-25T19:08:56.058 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105251.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105251.log.gz 2024-04-25T19:08:56.059 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42180.log: /var/log/ceph/ceph-client.admin.106450.log 2024-04-25T19:08:56.060 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-04-25T19:08:56.061 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-04-25T19:08:56.062 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106450.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106450.log.gz 2024-04-25T19:08:56.063 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39334.log 2024-04-25T19:08:56.064 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46581.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-04-25T19:08:56.068 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43492.log 2024-04-25T19:08:56.068 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39334.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39334.log.gz 2024-04-25T19:08:56.069 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41436.log 2024-04-25T19:08:56.069 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.0.43492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43492.log.gz 2024-04-25T19:08:56.070 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106031.log 2024-04-25T19:08:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41436.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41436.log.gz 2024-04-25T19:08:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103653.log 2024-04-25T19:08:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106031.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106031.log.gz 2024-04-25T19:08:56.073 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-04-25T19:08:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103653.log.gz 2024-04-25T19:08:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-04-25T19:08:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45909.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-04-25T19:08:56.077 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40854.log: /var/log/ceph/ceph-client.admin.39780.log 2024-04-25T19:08:56.078 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-04-25T19:08:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42360.log 2024-04-25T19:08:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39780.log.gz 2024-04-25T19:08:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44850.log 2024-04-25T19:08:56.082 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42360.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42360.log.gz 2024-04-25T19:08:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-04-25T19:08:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44850.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-04-25T19:08:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40457.log: /var/log/ceph/ceph-client.admin.41184.log 2024-04-25T19:08:56.086 INFO:teuthology.orchestra.run.smithi123.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-04-25T19:08:56.087 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44934.log 2024-04-25T19:08:56.088 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41184.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-04-25T19:08:56.089 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44934.log: -- /var/log/ceph/ceph-client.admin.42696.log 2024-04-25T19:08:56.090 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44934.log.gz 2024-04-25T19:08:56.091 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41607.log 2024-04-25T19:08:56.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42696.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-04-25T19:08:56.093 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-04-25T19:08:56.094 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41607.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41607.log.gz 2024-04-25T19:08:56.095 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-25T19:08:56.096 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40501.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-04-25T19:08:56.097 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105209.log/var/log/ceph/ceph-client.admin.44724.log: 2024-04-25T19:08:56.098 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-25T19:08:56.099 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41310.log 2024-04-25T19:08:56.100 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105209.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105209.log.gz 2024-04-25T19:08:56.101 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40728.log/var/log/ceph/ceph-client.admin.41310.log: 2024-04-25T19:08:56.102 INFO:teuthology.orchestra.run.smithi123.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41310.log.gz 2024-04-25T19:08:56.103 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T19:08:56.104 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40728.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40728.log.gz 2024-04-25T19:08:56.105 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-04-25T19:08:56.107 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44472.log 2024-04-25T19:08:56.109 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-04-25T19:08:56.111 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46707.log 2024-04-25T19:08:56.112 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40586.log 2024-04-25T19:08:56.113 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46707.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44472.log.gz 2024-04-25T19:08:56.114 INFO:teuthology.orchestra.run.smithi123.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46707.log.gz 2024-04-25T19:08:56.116 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41940.log 2024-04-25T19:08:56.119 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41940.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40586.log.gz 2024-04-25T19:08:56.120 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40205.log 2024-04-25T19:08:56.121 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41940.log.gz 2024-04-25T19:08:56.122 INFO:teuthology.orchestra.run.smithi146.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T19:08:56.123 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T19:08:56.123 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-04-25T19:08:56.125 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41952.log 2024-04-25T19:08:56.126 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45740.log: /var/log/ceph/ceph-client.admin.40205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-04-25T19:08:56.128 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.40205.log.gz 2024-04-25T19:08:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41952.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41952.log.gz 2024-04-25T19:08:56.131 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39376.log 2024-04-25T19:08:56.133 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106199.log 2024-04-25T19:08:56.134 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39376.log.gz 2024-04-25T19:08:56.135 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106199.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106199.log.gz 2024-04-25T19:08:56.136 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106283.log 2024-04-25T19:08:56.137 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104572.log 2024-04-25T19:08:56.139 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106283.log.gz 2024-04-25T19:08:56.140 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104572.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104572.log.gz 2024-04-25T19:08:56.141 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104530.log 2024-04-25T19:08:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105106.log 2024-04-25T19:08:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104530.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104530.log.gz 2024-04-25T19:08:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105106.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105106.log.gz 2024-04-25T19:08:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105821.log 2024-04-25T19:08:56.147 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-04-25T19:08:56.149 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105821.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105821.log.gz 2024-04-25T19:08:56.150 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42188.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-04-25T19:08:56.150 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-04-25T19:08:56.152 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-04-25T19:08:56.153 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41610.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-04-25T19:08:56.155 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48164.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-04-25T19:08:56.155 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-04-25T19:08:56.157 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2024-04-25T19:08:56.158 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41945.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-04-25T19:08:56.160 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39120.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2024-04-25T19:08:56.160 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40812.log 2024-04-25T19:08:56.162 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41394.log 2024-04-25T19:08:56.163 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40812.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40812.log.gz 2024-04-25T19:08:56.164 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41394.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41394.log.gz 2024-04-25T19:08:56.165 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104446.log 2024-04-25T19:08:56.167 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104404.log 2024-04-25T19:08:56.168 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104446.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104446.log.gz 2024-04-25T19:08:56.169 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104404.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104404.log.gz 2024-04-25T19:08:56.170 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2024-04-25T19:08:56.171 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-04-25T19:08:56.173 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47085.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2024-04-25T19:08:56.174 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.34071.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-04-25T19:08:56.175 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41950.log 2024-04-25T19:08:56.177 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41950.log.gz 2024-04-25T19:08:56.178 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46287.log 2024-04-25T19:08:56.179 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40770.log 2024-04-25T19:08:56.181 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46287.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46287.log.gz 2024-04-25T19:08:56.182 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40770.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40770.log.gz 2024-04-25T19:08:56.183 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39588.log 2024-04-25T19:08:56.184 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42318.log 2024-04-25T19:08:56.186 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39588.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39588.log.gz 2024-04-25T19:08:56.187 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42318.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42318.log.gz 2024-04-25T19:08:56.188 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47996.log 2024-04-25T19:08:56.189 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105293.log 2024-04-25T19:08:56.191 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47996.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-04-25T19:08:56.192 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105293.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105293.log.gz 2024-04-25T19:08:56.192 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45381.log 2024-04-25T19:08:56.194 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47839.log 2024-04-25T19:08:56.195 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45381.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45381.log.gz 2024-04-25T19:08:56.197 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47839.log.gz 2024-04-25T19:08:56.197 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45549.log 2024-04-25T19:08:56.199 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39462.log 2024-04-25T19:08:56.200 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45549.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45549.log.gz 2024-04-25T19:08:56.202 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39462.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39462.log.gz 2024-04-25T19:08:56.202 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-04-25T19:08:56.204 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47795.log 2024-04-25T19:08:56.205 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42486.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-04-25T19:08:56.207 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47795.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47795.log.gz 2024-04-25T19:08:56.207 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2024-04-25T19:08:56.209 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106408.log 2024-04-25T19:08:56.210 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41951.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-04-25T19:08:56.210 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105335.log 2024-04-25T19:08:56.212 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106408.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106408.log.gz 2024-04-25T19:08:56.213 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-04-25T19:08:56.213 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105335.log.gz 2024-04-25T19:08:56.215 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47169.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-04-25T19:08:56.216 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33623.log 2024-04-25T19:08:56.217 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43960.log 2024-04-25T19:08:56.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.33623.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.33623.log.gz 2024-04-25T19:08:56.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.2.43960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43960.log.gz 2024-04-25T19:08:56.220 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2024-04-25T19:08:56.222 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47950.log 2024-04-25T19:08:56.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz 2024-04-25T19:08:56.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz 2024-04-25T19:08:56.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47001.log 2024-04-25T19:08:56.227 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41613.log 2024-04-25T19:08:56.228 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47001.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47001.log.gz 2024-04-25T19:08:56.229 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41613.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41613.log.gz 2024-04-25T19:08:56.230 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41941.log 2024-04-25T19:08:56.232 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45018.log 2024-04-25T19:08:56.233 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41941.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41941.log.gz 2024-04-25T19:08:56.234 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45018.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45018.log.gz 2024-04-25T19:08:56.235 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46371.log 2024-04-25T19:08:56.237 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2024-04-25T19:08:56.238 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46371.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46371.log.gz 2024-04-25T19:08:56.238 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-04-25T19:08:56.239 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46539.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2024-04-25T19:08:56.240 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-04-25T19:08:56.242 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48332.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-04-25T19:08:56.243 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.47625.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46413.log 2024-04-25T19:08:56.243 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-04-25T19:08:56.244 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46083.log 2024-04-25T19:08:56.246 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46413.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46413.log.gz 2024-04-25T19:08:56.247 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-04-25T19:08:56.247 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46083.log.gz 2024-04-25T19:08:56.248 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38724.log 2024-04-25T19:08:56.249 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46125.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz 2024-04-25T19:08:56.250 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47329.log 2024-04-25T19:08:56.251 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.38724.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38724.log.gz 2024-04-25T19:08:56.252 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 2024-04-25T19:08:56.253 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47329.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47329.log.gz 2024-04-25T19:08:56.254 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-04-25T19:08:56.255 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41616.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gz 2024-04-25T19:08:56.257 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44178.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-04-25T19:08:56.257 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2024-04-25T19:08:56.259 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-04-25T19:08:56.261 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41226.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2024-04-25T19:08:56.261 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106115.log 2024-04-25T19:08:56.262 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44640.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-04-25T19:08:56.263 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-04-25T19:08:56.264 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106115.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106115.log.gz 2024-04-25T19:08:56.265 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44892.log 2024-04-25T19:08:56.266 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41100.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-04-25T19:08:56.268 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44892.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44892.log.gz 2024-04-25T19:08:56.268 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-04-25T19:08:56.270 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42444.log 2024-04-25T19:08:56.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44598.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-04-25T19:08:56.272 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-04-25T19:08:56.273 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42444.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42444.log.gz 2024-04-25T19:08:56.274 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105947.log 2024-04-25T19:08:56.275 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2024-04-25T19:08:56.276 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105947.log.gz 2024-04-25T19:08:56.277 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105989.log 2024-04-25T19:08:56.278 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-04-25T19:08:56.280 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105989.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105989.log.gz 2024-04-25T19:08:56.281 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105569.log 2024-04-25T19:08:56.281 INFO:teuthology.orchestra.run.smithi123.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-04-25T19:08:56.286 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105569.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105569.log.gz 2024-04-25T19:08:56.286 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-04-25T19:08:56.286 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-04-25T19:08:56.287 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44220.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-04-25T19:08:56.287 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-04-25T19:08:56.288 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46041.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-04-25T19:08:56.289 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-04-25T19:08:56.290 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45783.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-04-25T19:08:56.291 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47499.log: -- /var/log/ceph/ceph-mds.a.log 2024-04-25T19:08:56.292 INFO:teuthology.orchestra.run.smithi123.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-04-25T19:08:56.293 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46749.log 2024-04-25T19:08:56.296 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.46749.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40896.log 2024-04-25T19:08:56.296 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T19:08:56.296 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46749.log.gz 2024-04-25T19:08:56.298 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42528.log 2024-04-25T19:08:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40896.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40896.log.gz 2024-04-25T19:08:56.300 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log/var/log/ceph/ceph-client.admin.42528.log: 2024-04-25T19:08:56.301 INFO:teuthology.orchestra.run.smithi123.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42528.log.gz 2024-04-25T19:08:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66948.log 2024-04-25T19:08:56.303 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47541.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-04-25T19:08:56.303 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.66948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-04-25T19:08:56.304 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66948.log.gz 2024-04-25T19:08:56.306 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-04-25T19:08:56.307 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40938.log 2024-04-25T19:08:56.308 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45186.log 2024-04-25T19:08:56.310 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.40938.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40938.log.gz 2024-04-25T19:08:56.310 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45423.log 2024-04-25T19:08:56.311 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45186.log.gz 2024-04-25T19:08:56.312 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2024-04-25T19:08:56.313 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45423.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45423.log.gz 2024-04-25T19:08:56.314 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105737.log 2024-04-25T19:08:56.315 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39672.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2024-04-25T19:08:56.316 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2024-04-25T19:08:56.317 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105737.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105737.log.gz 2024-04-25T19:08:56.318 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39546.log 2024-04-25T19:08:56.320 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.47249.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2024-04-25T19:08:56.320 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67518.log 2024-04-25T19:08:56.320 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39546.log.gz 2024-04-25T19:08:56.322 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104834.log 2024-04-25T19:08:56.323 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.67518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67518.log.gz 2024-04-25T19:08:56.323 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105611.log 2024-04-25T19:08:56.324 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.104834.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104834.log.gz 2024-04-25T19:08:56.325 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42276.log 2024-04-25T19:08:56.327 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.105611.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105611.log.gz 2024-04-25T19:08:56.327 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42990.log 2024-04-25T19:08:56.328 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42276.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42276.log.gz 2024-04-25T19:08:56.329 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103724.log 2024-04-25T19:08:56.331 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42990.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42990.log.gz 2024-04-25T19:08:56.331 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103724.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103724.log.gz 2024-04-25T19:08:56.331 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-04-25T19:08:56.333 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-04-25T19:08:56.334 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44430.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-04-25T19:08:56.335 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-04-25T19:08:56.336 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48206.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-04-25T19:08:56.337 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42190.log 2024-04-25T19:08:56.338 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.41954.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-04-25T19:08:56.339 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-04-25T19:08:56.340 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.42190.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42190.log.gz 2024-04-25T19:08:56.340 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41618.log 2024-04-25T19:08:56.342 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44262.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-04-25T19:08:56.343 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.41618.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45999.log 2024-04-25T19:08:56.343 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41618.log.gz 2024-04-25T19:08:56.344 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2024-04-25T19:08:56.346 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45999.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45999.log.gz 2024-04-25T19:08:56.346 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-04-25T19:08:56.347 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.48122.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-04-25T19:08:56.348 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46791.log 2024-04-25T19:08:56.349 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44094.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-04-25T19:08:56.350 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-04-25T19:08:56.351 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46791.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46791.log.gz 2024-04-25T19:08:56.353 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.44136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-04-25T19:08:56.353 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-04-25T19:08:56.355 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106073.log 2024-04-25T19:08:56.356 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46665.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-04-25T19:08:56.357 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39714.log 2024-04-25T19:08:56.358 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.106073.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106073.log.gz 2024-04-25T19:08:56.360 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39714.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.39714.log.gz 2024-04-25T19:08:56.383 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T19:08:56.485 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T19:08:56.724 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T19:08:56.760 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T19:08:56.776 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T19:08:56.798 INFO:teuthology.orchestra.run.smithi146.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T19:08:56.801 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-04-25T19:08:56.801 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m0.899s 2024-04-25T19:08:56.802 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m3.357s 2024-04-25T19:08:56.802 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.378s 2024-04-25T19:08:56.848 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T19:08:56.958 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T19:08:56.963 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T19:08:56.984 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T19:08:56.988 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-04-25T19:08:56.988 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m1.092s 2024-04-25T19:08:56.988 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m4.329s 2024-04-25T19:08:56.989 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m1.044s 2024-04-25T19:08:56.990 INFO:tasks.ceph:Archiving logs... 2024-04-25T19:08:56.990 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi123/log 2024-04-25T19:08:56.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:08:57.807 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi146/log 2024-04-25T19:08:57.808 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:08:58.329 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T19:08:58.340 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T19:08:58.341 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:08:58.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:08:58.485 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T19:08:58.495 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T19:08:58.495 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:08:58.502 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:08:58.549 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:08:58.558 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:08:58.559 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:08:58.559 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-25T19:08:58.559 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +854us[ +854us] +/- 47ms 2024-04-25T19:08:58.559 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -1503us[-1504us] +/- 71ms 2024-04-25T19:08:58.559 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +160us[ +160us] +/- 80ms 2024-04-25T19:08:58.560 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:08:58.568 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:08:58.568 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T19:08:58.568 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +915us[ +916us] +/- 47ms 2024-04-25T19:08:58.568 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1467us[-1467us] +/- 71ms 2024-04-25T19:08:58.568 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +235us[ +236us] +/- 80ms 2024-04-25T19:08:58.569 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:08:58.570 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T19:08:58.579 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T19:08:58.580 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T19:08:58.589 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:08:58.789 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:08:58.982 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:08:59.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:08:59.070 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:08:59.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:08:59.164 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T19:08:59.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T19:08:59.190 INFO:teuthology.task.internal:Duration was 887.075908 seconds 2024-04-25T19:08:59.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T19:08:59.200 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T19:08:59.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:08:59.205 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:08:59.339 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:08:59.347 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:08:59.860 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T19:08:59.860 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-04-25T19:08:59.860 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:08:59.924 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T19:08:59.924 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:08:59.996 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T19:08:59.996 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:09:00.004 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:09:00.088 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T19:09:00.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:09:00.093 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:09:00.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T19:09:00.276 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T19:09:00.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:09:00.357 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:09:00.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T19:09:00.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:09:00.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:09:00.553 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-04-25T19:09:00.559 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-04-25T19:09:00.601 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:09:00.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:09:00.648 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:09:00.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:09:00.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T19:09:00.710 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T19:09:00.711 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi123 2024-04-25T19:09:00.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:09:00.807 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673292/remote/smithi146 2024-04-25T19:09:00.807 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:09:00.906 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T19:09:00.907 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:09:00.914 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:09:00.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T19:09:00.982 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T19:09:00.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T19:09:01.014 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T19:09:01.014 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:09:01.021 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:09:01.072 INFO:teuthology.orchestra.run.smithi123.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:09 /home/ubuntu/cephtest 2024-04-25T19:09:01.079 INFO:teuthology.orchestra.run.smithi146.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:09 /home/ubuntu/cephtest 2024-04-25T19:09:01.082 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T19:09:01.093 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T19:09:01.115 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 887.0759084224701 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T19:09:01.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:09:01.208 INFO:teuthology.run:pass